Information processing device, information processing method, and imaging device

Abstract

The present disclosure pertains to an information processing device, an information processing method, and an imaging device making it possible to execute two-dimensional code recognition processing in a shorter amount of time. In the invention, a two-dimensional code recognition processing unit performs two-dimensional code recognition processing of using a luminance image, which is outputted from an image processing unit of an image sensor via a route different from that of an RGB image, to recognize a two-dimensional code captured in the luminance image. The luminance image is outputted from the image processing unit if, for example, the capture of a two-dimensional code has been recognized. The present technology can be applied, for example, to an imaging device provided with a two-dimensional code recognition processing unit for detecting and decoding a two-dimensional code.

Description

Information processing device, information processing method, and imaging device

[0001] The present disclosure relates to an information processing device, an information processing method, and an imaging device, and more particularly to an information processing device, an information processing method, and an imaging device that are capable of executing two-dimensional code recognition processing in a shorter time.

[0002] Conventionally, two-dimensional codes have been used to allow IoT (Internet of Things) devices, network cameras, and the like to read various types of information. For example, development is underway on imaging devices equipped with two-dimensional code recognition processing units that detect and decode two-dimensional codes.

[0003] For example, Patent Documents 1 and 2 disclose a technique for converting a color image output from an image sensor into a grayscale image and recognizing a two-dimensional code captured by the image sensor.

[0004] JP-T-2014-535092 A JP-A-2015-99482 A

[0005] However, in IoT devices with limited computing resources, the processing time required for 2D code recognition is long, which is expected to reduce usability. For example, in a configuration where a color image of a 2D code is captured and transferred from an image sensor to a 2D code recognition processor, taking into account the transfer time for transferring the color image, a long processing time is required from the time the 2D code is captured until it is recognized.

[0006] The present disclosure has been made in view of such circumstances, and aims to enable two-dimensional code recognition processing to be performed in a shorter time.

[0007] An information processing device according to one aspect of the present disclosure includes a two-dimensional code recognition processing unit that performs two-dimensional code recognition processing using a luminance image output from an image processing unit of an image sensor via a route different from that of an RGB image, to recognize a two-dimensional code depicted in the luminance image.

[0008] An information processing method according to one aspect of the present disclosure includes an information processing device performing a two-dimensional code recognition process that uses a luminance image output from an image processing unit of an image sensor via a route different from that of an RGB image to recognize a two-dimensional code depicted in the luminance image.

[0009] An imaging device according to one aspect of the present disclosure includes an information processing device having an image processing unit of an image sensor that outputs a luminance image via a route different from that of an RGB image, and a two-dimensional code recognition processing unit that performs two-dimensional code recognition processing using the luminance image to recognize a two-dimensional code depicted in the luminance image.

[0010] In one aspect of the present disclosure, a two-dimensional code recognition process is performed in which a luminance image output from an image processing unit of an image sensor via a route different from that of the RGB image is used to recognize a two-dimensional code depicted in the luminance image.

[0011] Fig. 1 is a block diagram showing an example configuration of an embodiment of an imaging device to which the present technology is applied. Fig. 2 is a block diagram showing an example configuration of a two-dimensional code recognition processing unit. Fig. 3 is a diagram explaining shutter speed adjustment, contrast enhancement, and edge enhancement. Fig. 4 is a flowchart explaining two-dimensional code recognition processing. Fig. 5 is a diagram explaining an improvement result of the processing time required for two-dimensional code recognition processing. Fig. 6 is a block diagram showing an example configuration of an embodiment of a computer to which the present technology is applied.

[0012] Hereinafter, specific embodiments to which the present technology is applied will be described in detail with reference to the drawings.

[0013] <Configuration Example of Imaging Apparatus> FIG. 1 is a block diagram showing a configuration example of an embodiment of an imaging apparatus to which the present technology is applied.

[0014] As shown in FIG. 1, the imaging device 11 is configured with an imaging unit 21, an imaging control unit 22, an image processing unit 23, and an image recognition processing unit 24, and the image recognition processing unit 24 has a DNN (Deep Neural Network) processing unit 31 and a two-dimensional code recognition processing unit 32.

[0015] The imaging unit 21 is, for example, a sensor chip of an image sensor, and is configured with a plurality of pixels arranged in an array, and outputs an image obtained by capturing an image at an exposure and shutter speed controlled by the imaging control unit 22. The imaging unit 21 has, for example, pixels that receive red, green, and blue light arranged in a Bayer array, and supplies an image (hereinafter referred to as a RAW image) in which the colors of the light received by each pixel are arranged as they are to the image processing unit 23.

[0016] The imaging control unit 22 controls the exposure and shutter speed of the imaging unit 21 based on image quality setting instructions from the two-dimensional code recognition processing unit 32 of the image recognition processing unit 24. For example, the imaging control unit 22 controls the exposure of the imaging unit 21 to increase the shutter speed of the imaging unit 21 in order to capture a blur-free two-dimensional code (specifically, to change the shutter speed from 1 / 100 to 1 / 180), because the image becomes darker accordingly. Note that if the shutter speed is increased too much, noise will occur, so the imaging control unit 22 controls the shutter speed to an extent that will prevent the two-dimensional code from becoming unrecognizable due to noise.

[0017] The image processing unit 23 is, for example, an image signal processor (ISP) chip of an image sensor, and outputs an RGB image obtained by performing image processing such as demosaic processing on the RAW image supplied from the imaging unit 21. The image processing unit 23 also supplies to the image recognition processing unit 24 an image for DNN and a luminance image output via a route different from that of the RGB image. For example, the image processing unit 23 converts the RAW image supplied from the imaging unit 21 into an image for DNN suitable for DNN processing performed in the DNN processing unit 31 of the image recognition processing unit 24 and outputs the converted image. When the DNN processing unit 31 of the image recognition processing unit 24 instructs the image processing unit 23 to output a luminance image, the image processing unit 23 converts the RAW image supplied from the imaging unit 21 into a luminance image and outputs the converted image.

[0018] The image recognition processing unit 24 recognizes the subject shown in the image captured by the imaging unit 21 by having the DNN processing unit 31 perform DNN processing on the image for DNN supplied from the image processing unit 23, and outputs the resulting image recognition result. Also, the image recognition processing unit 24 recognizes the two-dimensional code shown in the image captured by the imaging unit 21 by having the two-dimensional code recognition processing unit 32 perform two-dimensional code recognition processing on the luminance image supplied from the image processing unit 23, and outputs the resulting two-dimensional code recognition result (data obtained by decoding the two-dimensional code).

[0019] The DNN processing unit 31 recognizes the subject depicted in the image captured by the imaging unit 21 by performing DNN processing on the image for DNN, and if it recognizes that a two-dimensional code is depicted in the image captured by the imaging unit 21, it instructs the image processing unit 23 to output a brightness image.

[0020] When the DNN processing unit 31 recognizes that a two-dimensional code is captured in the image captured by the imaging unit 21, the two-dimensional code recognition processing unit 32 starts two-dimensional code recognition processing using the luminance image supplied from the image processing unit 23. As will be described later, in the two-dimensional code recognition processing, the two-dimensional code recognition processing unit 32 can instruct the imaging control unit 22 to set the image quality so that the image quality is suitable for recognizing the two-dimensional code, and can perform image processing on the luminance image (for example, contrast enhancement, cropping, edge enhancement, etc.) suitable for detecting and decoding the two-dimensional code.

[0021] In this way, the imaging device 11 is configured so that, at the start of the two-dimensional code recognition process, a luminance image output from the image sensor via a route different from the RGB image is directly input to the two-dimensional code recognition processing unit 32. This allows the image transfer speed to be reduced compared to, for example, a configuration in which an RGB image is input to the two-dimensional code recognition processing unit 32, thereby shortening the processing time required from when the two-dimensional code is captured to when it is recognized. Furthermore, by performing the two-dimensional code recognition process on the luminance image, the imaging device 11 can reduce the size of the data to be processed, thereby also shortening the processing time required for the two-dimensional code recognition process.

[0022] Furthermore, the imaging device 11 can automatically switch processing to perform two-dimensional code recognition processing when a two-dimensional code is captured in an image captured by the imaging unit 21. When the two-dimensional code recognition processing unit 32 executes the two-dimensional code recognition processing, the imaging control unit 22 controls the exposure and shutter speed of the imaging unit 21 so that the image quality is suitable for recognizing the two-dimensional code, thereby enabling the two-dimensional code to be recognized more reliably.

[0023] Therefore, the imaging device 11 can execute the two-dimensional code recognition process in a shorter time.

[0024] FIG. 2 is a block diagram showing an example of the configuration of the two-dimensional code recognition processing unit 32. As shown in FIG.

[0025] As shown in FIG. 2, the two-dimensional code recognition processing unit 32 is configured to include an image quality setting instruction unit 41, a luminance image acquisition unit 42, a brightness determination unit 43, a contrast enhancement unit 44, a two-dimensional code detection and decoding unit 45, a trapezoid tilt detection unit 46, and a crop and edge enhancement unit 47.

[0026] When the 2D code recognition processing unit 32 starts the 2D code recognition process, the image quality setting instruction unit 41 instructs the imaging control unit 22 to set the image quality so that the image quality is suitable for recognizing the 2D code. For example, as shown on the left side of A in Fig. 3, if the shutter speed of the imaging unit 21 is slow, a blurred 2D code will be captured. Therefore, when the image quality setting instruction unit 41 instructs the imaging control unit 22 to set the image quality, the imaging unit 21 controls the imaging unit 21 to adjust the shutter speed to increase, allowing the imaging unit 21 to capture a blur-free 2D code, as shown on the right side of A in Fig. 3. Furthermore, when the image quality setting instruction unit 41 is notified by the 2D code detection and decoding unit 45 that the 2D code has been successfully decoded, it instructs the imaging control unit 22 to return the image quality setting.

[0027] The luminance image acquisition unit 42 acquires the luminance image supplied from the image processing unit 23 and supplies it to the brightness determination unit 43 .

[0028] The brightness determination unit 43 determines the brightness of the luminance image based on pixel values ​​in the central region of the luminance image supplied from the luminance image acquisition unit 42. If the brightness determination unit 43 determines that the luminance image is not sufficiently bright (for example, the pixel value is equal to or less than a predetermined threshold), it supplies the luminance image to the contrast enhancement unit 44. On the other hand, if the brightness determination unit 43 determines that the luminance image is sufficiently bright, it supplies the luminance image to the two-dimensional code detection and decoding unit 45. For example, the threshold for determining the brightness of the luminance image is adjusted so as to improve the recognition accuracy of the two-dimensional code.

[0029] The contrast enhancement unit 44 enhances the contrast of the luminance image supplied from the brightness determination unit 43 by converting pixel values ​​so that the histogram of the luminance image is generally averaged (flattened). For example, by enhancing the contrast of a luminance image that is insufficiently bright, as shown on the left side of B in Fig. 3, it is possible to obtain a luminance image in which the distinction between white and black areas is improved, as shown on the right side of B in Fig. 3. The contrast enhancement unit 44 then supplies the contrast-enhanced luminance image to the two-dimensional code detection and decoding unit 45.

[0030] The two-dimensional code detection and decoding unit 45 detects and decodes the two-dimensional code depicted in the luminance image supplied from the brightness determination unit 43 or the luminance image supplied from the contrast enhancement unit 44. For example, the two-dimensional code detection and decoding unit 45 can detect the two-dimensional code by recognizing markers placed on the two-dimensional code. The two-dimensional code detection and decoding unit 45 then decodes the two-dimensional code by referencing a library for decoding two-dimensional codes, and if the decoding of the two-dimensional code is successful, outputs the data obtained by decoding the two-dimensional code as the two-dimensional code recognition result.

[0031] On the other hand, if the two-dimensional code detection and decoding unit 45 does not succeed (fails) in decoding the two-dimensional code, it supplies the luminance image to the trapezoid tilt detection unit 46. Furthermore, if the two-dimensional code detection and decoding unit 45 is notified by the trapezoid tilt detection unit 46 that the two-dimensional code shown in the luminance image is not inclined to a trapezoid, it supplies the luminance image to the crop and edge enhancement unit 47.

[0032] The trapezoidal tilt detection unit 46 detects whether the two-dimensional code shown in the luminance image supplied from the two-dimensional code detection and decoding unit 45 is inclined to a trapezoid. If the trapezoidal tilt detection unit 46 detects that the two-dimensional code shown in the luminance image is inclined to a trapezoid, it notifies the outside of the imaging device 11 that the two-dimensional code is inclined with respect to the imaging surface of the imaging device 11. This results in, for example, displaying a message indicating that the two-dimensional code is inclined or outputting an error sound.

[0033] The cropping and edge enhancing unit 47 crops the area in which the two-dimensional code is captured from the luminance image supplied from the two-dimensional code detection and decoding unit 45, and enhances the edges of the two-dimensional code captured in that area. For example, the cropping and edge enhancing unit 47 can cut out the two-dimensional code from the luminance image according to the coordinates in which the two-dimensional code is captured, and can enhance the edges of the two-dimensional code using a 3×3 kernel.

[0034] For example, the cropping and edge enhancement unit 47 enhances the edges of a blurred two-dimensional code as shown on the left side of Fig. 3C, thereby obtaining a two-dimensional code with improved blurring as shown on the right side of Fig. 3C. The cropping and edge enhancement unit 47 then supplies the two-dimensional code that has been cropped from the luminance image and whose edges have been enhanced to the two-dimensional code detection and decoding unit 45, which then decodes the two-dimensional code again.

[0035] The two-dimensional code recognition processing unit 32 is configured in this way, and can more reliably recognize two-dimensional codes by enhancing contrast when the brightness of the luminance image containing the two-dimensional code is insufficient, or by cropping and enhancing edges when the two-dimensional code cannot be decoded successfully. This allows the two-dimensional code recognition processing unit 32 to shorten the processing time required for two-dimensional code recognition processing.

[0036] <Example of Information Processing> FIG. 4 is a flowchart illustrating the two-dimensional code recognition processing executed in the two-dimensional code recognition processing unit 32. As shown in FIG.

[0037] In step S11, the image quality setting instruction unit 41 instructs the image capture control unit 22 to set image quality suitable for recognizing the two-dimensional code. As a result, the image capture control unit 22 controls the exposure and shutter speed of the image capture unit 21, and the image capture unit 21 captures an image with that image quality.

[0038] In step S12 , the luminance image acquisition unit 42 acquires the luminance image supplied from the image processing unit 23 and supplies it to the brightness determination unit 43 .

[0039] In step S13, the brightness determination unit 43 determines whether the brightness image supplied from the brightness image acquisition unit 42 in step S12 is sufficiently bright.

[0040] In step S13, if the brightness determination unit 43 determines that the luminance image is not sufficiently bright, the process proceeds to step S14.

[0041] In step S14, the brightness determination unit 43 supplies the luminance image to the contrast enhancement unit 44, and the contrast enhancement unit 44 enhances the contrast of the luminance image and supplies it to the two-dimensional code detection and decoding unit 45.

[0042] After the process of step S14, the process proceeds to step S15. Alternatively, if it is determined in step S13 that the luminance image is sufficiently bright, the luminance image is supplied from the brightness determination unit 43 to the two-dimensional code detection and decoding unit 45, and the process proceeds to step S15.

[0043] In step S15, the two-dimensional code detection and decoding unit 45 detects and decodes the two-dimensional code that is displayed in the brightness image supplied from the brightness determination unit 43 in step S13 or the brightness image supplied from the contrast enhancement unit 44 in step S14.

[0044] In step S16, the two-dimensional code detection and decoding unit 45 determines whether or not the detection of the two-dimensional code has been successful.

[0045] If the two-dimensional code detection and decoding unit 45 determines in step S16 that it has not successfully detected the two-dimensional code, the process returns to step S12, and the same process is repeated thereafter. On the other hand, if the two-dimensional code detection and decoding unit 45 determines in step S16 that it has successfully detected the two-dimensional code, the process proceeds to step S17.

[0046] In step S17, the two-dimensional code detection and decoding unit 45 determines whether or not the two-dimensional code has been successfully decoded.

[0047] If the two-dimensional code detection and decoding unit 45 determines in step S17 that the decoding of the two-dimensional code has not been successful, the process proceeds to step S18.

[0048] In step S18, the two-dimensional code detection and decoding unit 45 supplies the luminance image to the trapezoidal tilt detection unit 46. The trapezoidal tilt detection unit 46 detects whether the two-dimensional code shown in the luminance image is tilted like a trapezoid, and determines whether the two-dimensional code is tilted like a trapezoid.

[0049] In step S18, if the trapezoidal tilt detection unit 46 determines that the two-dimensional code shown in the luminance image is tilted like a trapezoid, the process proceeds to step S19.

[0050] In step S19, a notification that the two-dimensional code is oblique to the imaging surface of the imaging device 11 is sent to the outside of the imaging device 11. Thereafter, the process returns to step S12, and the same processes are repeated thereafter.

[0051] On the other hand, if the trapezoidal tilt detection unit 46 determines in step S18 that the two-dimensional code shown in the luminance image is not tilted into a trapezoid, the process proceeds to step S20.

[0052] In step S20, the two-dimensional code detection and decoding unit 45 supplies the luminance image to the cropping and edge enhancing unit 47. The cropping and edge enhancing unit 47 crops the area in which the two-dimensional code is captured from the luminance image, enhances the edges of the two-dimensional code captured in that area, and supplies the cropped image to the two-dimensional code detection and decoding unit 45.

[0053] In step S21, the two-dimensional code detection and decoding unit 45 again decodes the two-dimensional code whose edges have been enhanced by being cropped from the luminance image supplied from the cropping and edge enhancing unit 47 in step S20.

[0054] In step S22, the two-dimensional code detection and decoding unit 45 determines whether the decoding of the two-dimensional code was successful, and if it determines that the detection of the two-dimensional code was not successful, the processing returns to step S12, and the same processing is repeated thereafter.

[0055] On the other hand, if the two-dimensional code detection and decoding unit 45 determines in step S22 that the detection of the two-dimensional code has been successful, the process proceeds to step S23. Also, if the two-dimensional code detection and decoding unit 45 determines in step S17 that the detection of the two-dimensional code has been successful, the process proceeds to step S23.

[0056] In step S23, the two-dimensional code detection and decoding unit 45 outputs the data obtained by decoding the two-dimensional code as the two-dimensional code recognition result. Then, the image quality setting instruction unit 41 instructs the imaging control unit 22 to return the image quality setting, and then the two-dimensional code recognition process ends.

[0057] The above-described two-dimensional code recognition process enables the imaging unit 21 to capture an image of the two-dimensional code at an exposure and shutter speed that provide image quality suitable for recognizing the two-dimensional code, thereby improving the accuracy of recognizing the two-dimensional code. Furthermore, the two-dimensional code recognition process can prevent processing loops due to failure to detect or decode the two-dimensional code by, for example, enhancing contrast when the brightness of the luminance image containing the two-dimensional code is insufficient, or by cropping and enhancing edges when decoding the two-dimensional code is unsuccessful.

[0058] Therefore, the imaging device 11 can detect and decode two-dimensional codes in a shorter time by shortening the processing time for two-dimensional code recognition processing and improving the recognition accuracy of two-dimensional codes, thereby improving usability.

[0059] The results of improving the processing time required for two-dimensional code recognition processing will be described with reference to FIG.

[0060] Figure 5 shows an example of the total processing time required to recognize a two-dimensional code per loop, and the processing time from the start of the frame until detection of the two-dimensional code fails. Note that the example of processing time shown in Figure 5 is the result obtained by executing two-dimensional code recognition processing on a system configured with specific hardware or software, and does not necessarily indicate that such results will be obtained.

[0061] When performing two-dimensional code recognition processing using an RGB image, the total processing time required to recognize a two-dimensional code per loop was 3.78 seconds, and the processing time from the start of the frame to failure to detect the two-dimensional code was 0.24 seconds.

[0062] When 2D code recognition was performed using an RGB image that had undergone image processing, the total processing time required to recognize the 2D code per loop was 3.80 + α seconds, and the processing time from the start of the frame to failure to detect the 2D code was 0.23 seconds. Here, processing time α is the time required for image processing, and is estimated to be, for example, 0.1 seconds for contrast enhancement, less than 0.01 seconds for trapezoid tilt detection, 0.08 to 0.13 seconds (depending on the size of the 2D code) for cropping and edge enhancement, and 0.23 to 3.5 seconds for re-decoding the 2D code.

[0063] When performing two-dimensional code recognition processing using a luminance image, the total processing time required to recognize a two-dimensional code per loop was 0.91 seconds, and the processing time from the start of the frame to failure to detect the two-dimensional code was 0.23 seconds. In this way, by using a luminance image, processing time can be significantly reduced compared to when using an RGB image.

[0064] When a 2D code recognition process was performed using a processed luminance image, the total processing time required to recognize a 2D code per loop was 0.93 + α seconds, and the processing time from the start of the frame to failure to detect the 2D code was 0.24 seconds. In this way, by using a processed luminance image, it is possible to significantly reduce processing time compared to using a processed RGB image.

[0065] When performing two-dimensional code recognition processing using float (single-precision floating-point numbers) with an image-processed luminance image, the total processing time required to recognize a two-dimensional code per loop was 0.93 + α seconds, and the processing time from the start of the frame to failure to detect the two-dimensional code was 0.24 seconds. In other words, even when using float, two-dimensional code recognition processing can be performed in a sufficiently short processing time. Therefore, it is not necessary to use higher-performance double (double-precision floating-point numbers) for two-dimensional code recognition processing. In other words, the imaging device 11 can efficiently perform calculations to decode two-dimensional codes by reducing the data size by using a luminance image, so two-dimensional code recognition processing can be performed in a short time even when using float.

[0066] As described above, the imaging device 11 may automatically switch processing so that the DNN processing unit 31 recognizes the two-dimensional code and performs two-dimensional code recognition processing, or the user may manually switch processing. Furthermore, the imaging device 11 may prepare multiple libraries, and the two-dimensional code detection and decoding unit 45 may dynamically select a library to refer to when decoding the two-dimensional code according to the characteristics of the luminance image in which the two-dimensional code is captured.

[0067] <Example of Computer Configuration> Next, the above-described series of processes (information processing method) can be performed by hardware or software. When the series of processes is performed by software, a program constituting the software is installed in a general-purpose computer or the like.

[0068] FIG. 6 is a block diagram showing an example of the hardware configuration of a computer that executes the above-described series of processes by a program.

[0069] In the computer, a CPU (Central Processing Unit) 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, and an EEPROM (Electronically Erasable and Programmable Read Only Memory) 104 are interconnected by a bus 105. An input / output interface 106 is further connected to the bus 105, and the input / output interface 106 is connected to the outside.

[0070] In a computer configured as described above, the CPU 101 performs the above-described series of processes by loading programs stored in, for example, the ROM 102 and EEPROM 104 into the RAM 103 via the bus 105 and executing the programs. In addition, the programs executed by the computer (CPU 101) can be written in advance in the ROM 102, or can be installed or updated in the EEPROM 104 from outside via the input / output interface 106.

[0071] In this specification, the processing performed by a computer according to a program does not necessarily have to be performed in chronological order according to the order described in the flowchart. In other words, the processing performed by a computer according to a program also includes processing that is executed in parallel or individually (for example, parallel processing or object-based processing).

[0072] The program may be processed by a single computer (processor), or may be distributed among multiple computers. Furthermore, the program may be transferred to and executed on a remote computer.

[0073] Furthermore, in this specification, a system refers to a collection of multiple components (devices, modules (components), etc.), regardless of whether all of the components are contained in the same housing. Therefore, multiple devices housed in separate housings and connected via a network, and a single device housed in a single housing with multiple modules, are both systems.

[0074] Also, for example, a configuration described as one device (or processing unit) may be divided and configured as multiple devices (or processing units). Conversely, configurations described above as multiple devices (or processing units) may be combined and configured as one device (or processing unit). Of course, configurations other than those described above may be added to the configuration of each device (or each processing unit). Furthermore, as long as the configuration and operation of the entire system are substantially the same, part of the configuration of one device (or processing unit) may be included in the configuration of another device (or other processing unit).

[0075] Furthermore, for example, the present technology can be configured as a cloud computing system in which a single function is shared and processed collaboratively by a plurality of devices via a network.

[0076] Furthermore, for example, the above-described program can be executed in any device, as long as the device has the necessary functions (functional blocks, etc.) and can obtain the necessary information.

[0077] Also, for example, each step described in the above flowchart can be executed by one device or can be shared and executed by multiple devices. Furthermore, if one step includes multiple processes, the multiple processes included in that one step can be executed by one device or can be shared and executed by multiple devices. In other words, multiple processes included in one step can be executed as multiple step processes. Conversely, processes described as multiple steps can be executed collectively as a single step.

[0078] In addition, the processing of the steps of a program executed by a computer may be executed in chronological order according to the order described in this specification, or may be executed in parallel or individually at the required timing, such as when a call is made. In other words, as long as no contradiction occurs, the processing of each step may be executed in an order different from the order described above. Furthermore, the processing of the steps of this program may be executed in parallel with the processing of another program, or may be executed in combination with the processing of another program.

[0079] It should be noted that the present technologies described in this specification can be implemented independently and singly, unless a contradiction arises. Of course, any two or more of the present technologies can also be implemented in combination. For example, part or all of the present technologies described in any embodiment can be implemented in combination with part or all of the present technologies described in other embodiments. Furthermore, part or all of any of the present technologies described above can also be implemented in combination with other technologies not described above.

[0080] <Examples of Combinations of Configurations> The present technology can also be configured as follows. (1) An information processing device comprising: a two-dimensional code recognition processing unit that performs two-dimensional code recognition processing that uses a luminance image output from an image processing unit of an image sensor via a route different from an RGB image to recognize a two-dimensional code captured in the luminance image. (2) The information processing device described in (1) above, wherein the two-dimensional code recognition processing unit has a two-dimensional code detection and decoding unit that detects and decodes the two-dimensional code and outputs data obtained by decoding the two-dimensional code as a two-dimensional code recognition result. (3) The information processing device described in (1) or (2) above, wherein the two-dimensional code recognition processing unit further has an image quality setting instruction unit that, at the start of the two-dimensional code recognition processing, instructs the image sensor to set an image quality suitable for recognizing the two-dimensional code. (4) The information processing device according to (2) or (3) above, wherein the two-dimensional code recognition processing unit further has a brightness determination unit that determines the brightness of the luminance image, and a contrast enhancement unit that enhances the contrast of the luminance image when the brightness of the luminance image is equal to or lower than a predetermined threshold, and the two-dimensional code detection and decoding unit detects and decodes the two-dimensional code that is captured in the luminance image whose contrast has been enhanced by the contrast enhancement unit. (5) The information processing device according to (4) above, wherein the two-dimensional code recognition processing unit further has a trapezoid tilt detection unit that detects whether the two-dimensional code is tilted like a trapezoid when the two-dimensional code detection and decoding unit succeeds in detecting the two-dimensional code but fails to decode the two-dimensional code. (6) The information processing device described in (5) above, wherein the two-dimensional code recognition processing unit further includes a cropping and edge enhancement unit that, when the trapezoid tilt detection unit detects that the two-dimensional code is not tilted trapezoidally, crops an area in which the two-dimensional code is captured from the luminance image and emphasizes the edges of the two-dimensional code captured in that area, and the two-dimensional code detection and decoding unit again decodes the two-dimensional code that has been cropped from the luminance image and has its edges emphasized.(7) The information processing device according to any one of (1) to (6) above, wherein the two-dimensional code recognition processing unit performs the two-dimensional code recognition processing using single-precision floating-point numbers. (8) The information processing device according to any one of (1) to (7) above, further comprising an image recognition processing unit that recognizes a subject captured in an image output from the image processing unit, wherein the image recognition processing unit instructs the image processing unit to output the luminance image when it recognizes that the two-dimensional code is captured in the image. (9) The information processing device according to (8) above, wherein the image processing unit converts a RAW image into an image suitable for image recognition processing by the image recognition processing unit and supplies the image recognition processing unit to the image recognition processing unit. (10) The information processing device according to (2) above, wherein the two-dimensional code detection and decoding unit dynamically selects a library to be referenced for decoding the two-dimensional code from a plurality of libraries according to characteristics of the luminance image in which the two-dimensional code is captured. (11) An information processing method including: an information processing device performing two-dimensional code recognition processing to recognize a two-dimensional code captured in a luminance image output from an image processing unit of an image sensor via a route different from that of an RGB image, (12) An imaging device equipped with an information processing device having: an image processing unit of an image sensor that outputs a luminance image via a route different from that of an RGB image, and a two-dimensional code recognition processing unit that performs two-dimensional code recognition processing to recognize a two-dimensional code captured in the luminance image, using the luminance image.

[0081] It should be noted that the present embodiment is not limited to the above-described embodiment, and various modifications are possible within the scope of the gist of the present disclosure. Furthermore, the effects described in this specification are merely examples and are not intended to be limiting, and other effects may also be obtained.

[0082] DESCRIPTION OF SYMBOLS 11 Imaging device, 21 Imaging unit, 22 Imaging control unit, 23 Image processing unit, 24 Image recognition processing unit, 31 DNN processing unit, 32 Two-dimensional code recognition processing unit, 41 Image quality setting instruction unit, 42 Luminance image acquisition unit, 43 Brightness determination unit, 44 Contrast enhancement unit, 45 Two-dimensional code detection and decoding unit, 46 Trapezoidal tilt detection unit, 47 Cropping and edge enhancement unit

Claims

1. An information processing device equipped with a two-dimensional code recognition processing unit that performs two-dimensional code recognition processing to recognize a two-dimensional code depicted in a luminance image output from the image processing unit of an image sensor via a route different from that of an RGB image.

2. The information processing device according to claim 1, wherein the two-dimensional code recognition processing unit has a two-dimensional code detection and decoding unit that detects and decodes the two-dimensional code and outputs the data obtained by decoding the two-dimensional code as the two-dimensional code recognition result.

3. The information processing device according to claim 1, wherein the two-dimensional code recognition processing unit further has an image quality setting instruction unit that instructs the image sensor to set image quality at the start of the two-dimensional code recognition process so that the image quality is suitable for recognizing the two-dimensional code.

4. The information processing device according to claim 2, wherein the two-dimensional code recognition processing unit further comprises a brightness determination unit that determines the brightness of the luminance image, and a contrast enhancement unit that enhances the contrast of the luminance image when the brightness of the luminance image is equal to or less than a predetermined threshold, and the two-dimensional code detection and decoding unit detects and decodes the two-dimensional code that appears in the luminance image whose contrast has been enhanced by the contrast enhancement unit.

5. The information processing device according to claim 4, wherein the two-dimensional code recognition processing unit further includes a trapezoid tilt detection unit that detects whether the two-dimensional code is tilted like a trapezoid when the two-dimensional code detection and decoding unit successfully detects the two-dimensional code but fails to decode the two-dimensional code.

6. The information processing device according to claim 5, wherein the two-dimensional code recognition processing unit further includes a cropping and edge enhancement unit that, when the trapezoid tilt detection unit detects that the two-dimensional code is not tilted trapezoidally, crops the area in which the two-dimensional code is captured from the luminance image and emphasizes the edges of the two-dimensional code captured in that area, and the two-dimensional code detection and decoding unit again decodes the two-dimensional code that has been cropped from the luminance image and has its edges emphasized.

7. The information processing device according to claim 1, wherein the two-dimensional code recognition processing unit executes the two-dimensional code recognition process using single-precision floating-point numbers.

8. The information processing device according to claim 1, further comprising an image recognition processing unit that recognizes a subject captured in an image output from the image processing unit, wherein the image recognition processing unit instructs the image processing unit to output the luminance image when it recognizes that the two-dimensional code is captured in the image.

9. The information processing device according to claim 8, wherein the image processing unit converts the RAW image into an image suitable for image recognition processing by the image recognition processing unit and supplies the converted image to the image recognition processing unit.

10. The information processing device according to claim 2, wherein the two-dimensional code detection and decoding unit dynamically selects a library to be referenced for decoding the two-dimensional code from a plurality of libraries depending on the characteristics of the luminance image in which the two-dimensional code is captured.

11. An information processing method including an information processing device performing two-dimensional code recognition processing to recognize a two-dimensional code depicted in a luminance image output from the image processing unit of an image sensor via a route different from the RGB image.

12. An imaging device equipped with an information processing device having an image processing unit of an image sensor that outputs a luminance image via a route different from that of an RGB image, and a two-dimensional code recognition processing unit that performs two-dimensional code recognition processing using the luminance image to recognize a two-dimensional code captured in the luminance image.

Images