Character recognition device, character recognition method, and character recognition system

Abstract

This character recognition device is provided with: a recognition unit that recognizes a character string from a captured image including a trailer captured by an imaging device; and a post-processing unit that performs a horizontal writing concatenation process to concatenate two character strings, among the character strings recognized by the recognition unit, that satisfy a horizontal writing concatenation condition for concatenating the two character strings of horizontal writing arranged in a horizontal direction in the captured image, and outputs the recognized character string including the concatenated character string.

Description

Character recognition device, character recognition method, and character recognition system

[0001] The present disclosure relates to a character recognition device, a character recognition method, and a character recognition system.

[0002] Patent Literature 1 discloses a license plate recognition device that detects multiple quadrangles of license plate area candidates from an input image and performs character recognition on character areas included in the license plate area candidates. The license plate recognition device selects a license plate area candidate to output from the multiple detected license plate area candidates based on the character recognition results and quadrangle information for each license plate area candidate. The license plate recognition device then outputs information about the selected license plate area candidate.

[0003] Japanese Patent Application Laid-Open No. 2008-217347

[0004] In recent years, logistics using trailers has become increasingly common, leading to a growing need to efficiently manage the location of each trailer. Here, a trailer refers to a towed vehicle towed by a towing vehicle (also called a tractor). Trailers may carry containers for storing various items, etc. Generally, trailers or containers are marked with an identification (hereinafter referred to as "ID") for identifying the trailer or container. Therefore, it is believed that trailers or containers can be efficiently managed by recognizing this ID from captured images of vehicles, etc. However, captured images may contain various characters, and a successfully recognized character string may not necessarily be the desired ID for identifying the trailer or container. Therefore, when an imaged object, such as a trailer or container, contains multiple character strings of various sizes, colors, and designs, there is a need to detect the ID for identifying the trailer or container from among the multiple character strings and identify the imaged object. Hereinafter, the ID for identifying the trailer will be referred to as the "trailer ID," and the ID for identifying the container will be referred to as the "container ID."

[0005] In Patent Document 1, a quadrilateral that meets certain conditions is determined to be a candidate quadrilateral for the license plate area. However, since there are currently no unified standards for trailer IDs, the position, size, number of characters, etc. are not fixed. Therefore, even if a technology similar to Patent Document 1 is applied, it is difficult to detect the trailer ID from among the various character strings written at any position on the imaged object. Furthermore, multiple standards exist for container IDs, and like trailer IDs, it is difficult to detect them from among the various character strings.

[0006] The present disclosure has been devised in view of the above-described conventional situation, and aims to detect a character string that is considered to be a desired ID related to trailer identification from among a plurality of character strings.

[0007] The present disclosure provides a character recognition device including a recognition unit that recognizes character strings from an image captured by an imaging device, including a trailer; and a post-processing unit that performs horizontal combining processing to combine two character strings recognized by the recognition unit that satisfy a horizontal combining condition for combining two horizontally written character strings that are aligned horizontally in the captured image, and outputs the recognized character strings including the combined character string.

[0008] The present disclosure also provides a character recognition method that recognizes character strings from an image captured by an imaging device, including a trailer, performs a horizontal combining process to combine two recognized character strings that satisfy a horizontal combining condition for combining two character strings that are aligned horizontally in the captured image, and outputs the recognized character strings that include the combined character string.

[0009] The present disclosure also provides a character recognition system including at least one imaging device, a recognition unit that recognizes character strings from an image captured by the imaging device, including a trailer, and a post-processing unit that performs a horizontal combining process to combine two character strings recognized by the recognition unit that satisfy a horizontal combining condition for combining two character strings that are aligned horizontally in the captured image, and outputs the recognized character strings including the combined character string.

[0010] Any combination of the above components, and conversion of the expression of the present disclosure into a method, device, system, storage medium, computer program, etc., are also valid aspects of the present disclosure.

[0011] According to the present disclosure, it is possible to detect a character string that is considered to be a desired ID for identifying a trailer from among a plurality of character strings.

[0012] Schematic diagram showing an example of a character string included in a trailer Schematic diagram showing an example of a character string included in a trailer Block diagram showing an example of the configuration of a character recognition system according to embodiment 1 Flowchart of overall processing of a character recognition device according to embodiment 1 Schematic diagram for explaining detection masks and non-detection masks according to embodiment 1 Table diagram for explaining various conditions for combining processes according to embodiment 1 Schematic diagram for explaining horizontal writing combining process according to embodiment 1 Schematic diagram for explaining two-line combining process according to embodiment 1 Schematic diagram for explaining two-line combining process according to embodiment 1 Schematic diagram for explaining three-line combining process according to embodiment 1

[0013] Hereinafter, embodiments will be described in detail with reference to the drawings as appropriate. However, more detailed description than necessary may be omitted. For example, detailed description of well-known matters or redundant description of substantially identical configurations may be omitted. This is to avoid unnecessary redundancy in the following description and to facilitate understanding by those skilled in the art. Note that the accompanying drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims.

[0014] (Embodiment 1) In this embodiment, an example will be described in which a character string that is considered to be a trailer ID is detected from among multiple character strings included in a trailer. Note that the "character string that is considered to be a trailer ID" is not limited to a trailer ID, but may be read as an "ID related to trailer identification." For example, if a container ID is used to identify a trailer, the "character string that is considered to be a trailer ID" may include not only the trailer ID but also the container ID. First, with reference to Figures 1 and 2, an example of a character string included in a trailer TR will be described. Figure 1 is a schematic diagram showing an example of a character string included in a trailer TR. Note that Figure 1 is a schematic diagram of the trailer TR as viewed from the front right side.

[0015] The trailer TR is the entire object towed by the tractor LE. Generally, a vehicle towed by a tractor is called a trailer. A trailer may be configured as a set of a container that stores cargo but does not have a mechanism for movement such as wheels, and a dolly on which the container is loaded. A trailer may also be configured as a vehicle in which a portion for storing cargo and a mechanism for movement are integrated. When a tractor tows a set of a container and a dolly, only the dolly may be called a trailer. However, in this specification, for the sake of convenience, the entire object towed by the tractor, that is, both the set of the dolly and the container, and the vehicle equipped with a portion for storing cargo and a mechanism for movement, are called trailers.

[0016] In addition to the trailer ID, the trailer TR may contain a container ID, a company name, a telephone number, a number indicating the height of the trailer TR, a Uniform Resource Locator (hereinafter referred to as "URL"), a word, a text string, etc. Note that the strings contained in the trailer TR are not limited to the above-mentioned examples.

[0017] The character strings ST1, ST2, ST3, and ST4 are character strings indicating company names, and are each decorated with italic characters.

[0018] The character string ST5 is a character string expressed in English words, and is written below the character string ST2 indicating the company name.

[0019] The character strings ST6 and ST7 are character strings relating to numbers that identify the tractor LE.

[0020] The character string ST8 is a character string indicating the trailer ID. The character string ST8 is written vertically on a corner of the trailer TR (i.e., the boundary part of the surface that forms the housing of the trailer TR, which is located on the side of the trailer TR and is long in the direction of gravity).

[0021] 2 is a schematic diagram showing an example of a character string included in a trailer TR, as viewed from the rear of the trailer TR.

[0022] The character string ST9 is a character string indicating a trailer ID. The character string ST9 is the same as the character string ST8, but written horizontally.

[0023] The character string ST10 is a character string indicating a container ID. The character string ST10 is a character string consisting of English letters and numbers. In this specification, for the sake of simplicity, an example is shown in which both the trailer ID and the container ID are written in one trailer, but only one of the trailer ID and the container ID may be written in the trailer.

[0024] The character string ST11 is a character string indicating a telephone number. The character string ST11 is a character string represented by numbers.

[0025] The character string ST12 is a character string indicating the number of the license plate.

[0026] In this way, the trailer TR contains a variety of character strings.

[0027] In recent years, with the increase in logistics, there has been a growing need to efficiently manage a large number of trailers. For example, at locations where multiple trailers gather (e.g., a starting point, a relay point, or a destination point), it is necessary to identify each trailer. However, as shown in Figures 1 and 2, each trailer has multiple character strings written on it, making it difficult to find a desired ID, such as a trailer ID, from among the multiple character strings.

[0028] Next, a configuration example of the character recognition system 1 according to the first embodiment will be described with reference to Fig. 3. Fig. 3 is a block diagram showing a configuration example of the character recognition system 1 according to the first embodiment.

[0029] The character recognition system 1 includes at least an imaging device 10 , a display device 20 , and a character recognition device 30 .

[0030] The imaging device 10 is a device that captures images of trailers. The imaging device 10 is installed, for example, at the entrance to an area (hereinafter also referred to as a "yard") where cargo transported by tractors and trailers is loaded and unloaded or transferred, and where trailers towed by tractors or containers loaded on trailers are changed, and captures images of trailers and tractors entering and leaving the yard. The imaging device 10 is configured to include at least a lens (not shown) and an image sensor (not shown) as optical elements. The lens receives light reflected by an object within the angle of view of the area captured by the imaging device 10 and forms an optical image of the object on the light-receiving surface of the image sensor (in other words, the imaging surface). The image sensor is a solid-state imaging element such as a Charged Coupled Device (hereinafter referred to as "CCD") or a Complementary Metal Oxide Semiconductor (hereinafter referred to as "CMOS"). The image sensor converts an optical image formed on an imaging surface via a lens into an electrical signal every predetermined time (e.g., 1 / 30 (second)). For example, if the predetermined time is 1 / 30 (second), the frame rate of the imaging device 10 is 30 fps. The imaging device 10 may also generate image data or video data by performing predetermined signal processing on the electrical signal every predetermined time. Note that the image data is a still image, and the video data is a moving image. The imaging device 10 outputs the generated image data or video data to the character recognition device 30. Hereinafter, the image data and video data will be referred to as a captured image. 3 shows one imaging device 10, the character recognition system 1 may be configured to include multiple imaging devices. When the character recognition system 1 is configured to include multiple imaging devices, the multiple imaging devices are preferably positioned so that they can capture images of the trailer from multiple angles.

[0031] The display device 20 displays, for example, character strings output from the character recognition device 30. The display device 20 is a terminal equipped with a display unit, such as a personal computer (hereinafter referred to as a "PC"), a tablet, or a mobile phone. The display device 20 may be integrated with the character recognition device 30. While the example of FIG. 3 shows a single display device 20, the character recognition system 1 may include multiple display devices. When the character recognition system 1 includes multiple display devices, the display devices may have different forms and users. For example, the character strings output from the character recognition device 30 may be displayed on a PC or the like located in an office at the yard and on a tablet or the like carried by a worker. In this case, the level of detail of information displayed by each display device may be changed depending on the performance of each display device or the task required of the user of each display device. Furthermore, the display device 20 may not be provided in the first embodiment. This is because, for example, in cases where an external device (not shown) further processes the character string output from the character recognition device 30, the user does not necessarily need to visually check the character string output from the character recognition device 30.

[0032] The character recognition device 30 includes at least a processor 31, a memory 32, and a communication interface 33. The character recognition device 30 is, for example, a computing device such as a PC. Although the first embodiment describes an example in which the character recognition device 30 is configured as a single computing device, the character recognition device 30 may be configured such that a plurality of devices cooperate to realize its functions, or may be realized on the cloud.

[0033] The processor 31 is configured using, for example, a central processing unit (hereinafter referred to as "CPU"), a digital signal processor (hereinafter referred to as "DSP"), or a field programmable gate array (hereinafter referred to as "FPGA"). The processor 31 functions as a controller that manages the overall operation of the character recognition device 30. The processor 31 performs control processing for overseeing the operation of each unit of the character recognition device 30, data input / output processing between each unit of the character recognition device 30, data arithmetic processing, and data storage processing. The processor 31 operates according to a program stored in the memory 32. The processor 31 uses the memory 32 during operation and temporarily stores data generated or acquired by the processor 31 in the memory 32. The processor 31 realizes the functions of the character recognition unit 31A and the post-processing unit 31B by using the programs and data stored in the memory 32.

[0034] The character recognition unit 31A, which is an example of a recognition unit, detects a character string from the captured image acquired from the imaging device 10. The character recognition unit 31A may detect the character string using a known technique such as machine learning. The character recognition unit 31A then performs character recognition on the detected character string. The character recognition unit 31A may perform character recognition on the character string using a known technique such as Optical Character Recognition (hereinafter referred to as "OCR"). In the first embodiment, the character recognition is performed by OCR.

[0035] The post-processing unit 31B performs various processes described below on one or more character strings recognized by the character recognition unit 31A, enabling the character recognition device 30 to output a character string that is considered to be a trailer ID from the character strings contained in the captured image.

[0036] The determination of whether a certain character string is a trailer ID may be performed by an external device (not shown) other than the character recognition device 30, or may be performed by the processor 31 and memory 32 of the character recognition device 30 working together. In the first embodiment, the determination of whether a certain character string is a trailer ID is performed by an external device (not shown) other than the character recognition device 30 (hereinafter, for the sake of explanation, also referred to as a "determination device"). Because a captured image contains a wide variety of character strings, if the determination device determines whether each of the multiple character strings is a trailer ID, for example, this may result in a high processing load or a decrease in accuracy. Therefore, the character recognition device 30 outputs a character string that is considered to be a trailer ID from among the multiple character strings included in the captured image to the external device. This makes it possible to detect a character string that is considered to be a trailer ID from among the multiple character strings included in the captured image.

[0037] The communication interface 33 is an interface circuit for the character recognition device 30 to communicate wirelessly or wired with the imaging device 10 and the display device 20. The communication interface 33 may perform communication via a network 40. Communication methods used by the communication interface 33 include, for example, a wide area network (hereinafter referred to as "WAN"), a local area network (hereinafter referred to as "LAN"), long term evolution (hereinafter referred to as "LTE"), mobile communications such as 4G and 5G, power line communications, short-range wireless communications (e.g., Bluetooth (registered trademark) communications), and communications for mobile phones. The communication interface 33 may also communicate with an external device (not shown) via the network 40.

[0038] The memory 32 is configured using, for example, a random access memory (hereinafter referred to as "RAM") and a read only memory (hereinafter referred to as "ROM"), and stores and holds programs, data, etc. necessary for the operation of the character recognition device 30. The memory 32 also temporarily holds data generated during the operation of the character recognition device 30. The RAM is, for example, a work memory used during the operation of the character recognition device 30. The ROM, for example, stores and holds in advance programs for controlling the character recognition device 30.

[0039] Next, a flowchart of processing executed by the character recognition device 30 will be described with reference to Fig. 4. Fig. 4 is a flowchart of the overall processing of the character recognition device 30 according to the first embodiment. Each process in the flowchart according to Fig. 4 is executed by the processor 31. At the start of this flowchart, the character recognition device 30 has acquired a captured image from the imaging device 10. For the sake of convenience, it is assumed that the captured image includes multiple character strings.

[0040] When the captured image acquired from the imaging device 10 contains an image that appears to be a character string, the character recognition unit 31A detects at least one character string from the captured image and performs character recognition on the character string (step St100). For convenience of explanation, it is assumed that multiple character strings are recognized.

[0041] The post-processing unit 31B performs character size filtering on the character strings recognized in step St100 (step St101). The post-processing unit 31B excludes character strings recognized in step St100 that contain characters outside a specified size range from the targets of subsequent processing. The reason for filtering by character size is that trailer IDs and the like are intended to be visible to workers and are therefore unlikely to be written in extremely small characters. Furthermore, unlike advertisements and the like, trailer IDs and the like are character strings that do not need to be conspicuous and are therefore unlikely to be written in extremely large characters. Another reason for filtering by character size is that extremely small or large character sizes are likely to result in misrecognition of characters. A supplementary explanation of character size filtering will be provided later with reference to FIG. 5 .

[0042] The post-processing unit 31B performs filtering based on the detection mask or non-detection mask on the character strings that were not excluded in the processing of step St101 (step St102). As a result, character strings that are included (or not included) in a specified area of ​​the captured image are excluded from the targets of each subsequent processing. Details will be described later with reference to FIG. 5.

[0043] The post-processing unit 31B performs horizontal-writing combining processing on two character strings that satisfy the horizontal-writing combining condition among the character strings that were not excluded in the processing of step St102 (step St103). Simply put, the post-processing unit 31B combines two character strings that are aligned horizontally in the captured image by the horizontal-writing combining processing so that they can be treated as a single character string. Details will be described later with reference to FIGS. 6 and 7.

[0044] The post-processing unit 31B executes a three-line join process on three character strings that satisfy the three-line join condition among the character strings that were not joined in the process of step St103 (step St104). Simply put, the post-processing unit 31B joins the character strings split into three lines so that they can be treated as a single character string. Details will be described later with reference to FIGS. 6 and 10.

[0045] The post-processing unit 31B performs two-line joining processing on two character strings that satisfy the two-line joining conditions among the character strings that were not joined in the processing of step St104 (step St105). Simply put, the post-processing unit 31B joins two character strings that have been split into two lines so that they can be treated as a single character string. Details will be described later with reference to FIGS. 6, 8, and 9. Note that if three character strings satisfy the conditions for three-line joining processing, the top two or bottom two of the three character strings are likely to also satisfy the conditions for two-line joining processing. Therefore, in this embodiment, the three-line joining processing is configured to be performed before the two-line joining processing.

[0046] The post-processing unit 31B performs output restriction processing on character strings not excluded in the processing of step St102, including character strings combined in the various combining processes of step St102, step St103, or step St104 (step St106). In the processing of step St107 described below, the post-processing unit 31B outputs character strings not excluded in the processing of step St102, including character strings combined in the various combining processes of step St102, step St103, or step St104, to, for example, the display device 20 or a determination device (not shown). By the output restriction processing of step St106, a character string identical to a character string once output in the processing of step St107 is excluded from the processing of step St107 if a specified time has not elapsed since the previous output. In other words, if the post-processing unit 31B outputs a character string in the processing of subsequent step St107 by the output restriction processing of step St106, it prohibits the output of a character string identical to the character string until a specified time has elapsed since the output. This is to prevent the same character string from being repeatedly output and detected, for example, when the trailer continues to be included in the angle of view of the imaging device 10 for a certain period of time.

[0047] The post-processing unit 31B outputs the character strings that can be output to the display device 20 or a determination device (step St107). The character strings that can be output are character strings that were not excluded in the processing of step St102 and whose output was not restricted in step St106, including character strings combined in various combining processes of step St102, step St103, or step St104.

[0048] The processor 31 of the character recognition device 30 executes step St107 and ends this processing flow.

[0049] Next, filtering of character strings based on a detection mask or a non-detection mask in step St102 of the flowchart shown in Fig. 4 will be described with reference to Fig. 5. Fig. 5 is a schematic diagram for explaining a detection mask and a non-detection mask according to the first embodiment.

[0050] 5 shows a captured image 50 including character strings ST13 and ST14. For ease of explanation, trailers and the like are not shown in the captured image 50. The character string ST13 is "123456," and the character string ST14 is "123627." Here, the character strings ST13 and ST14 have each been detected by the character recognition unit 31A.

[0051] First, character string filtering based on a detection mask will be described. The user may set in advance an area in the captured image to which the detection mask is to be applied. In the example of Fig. 5, a detection mask 52 is applied to an area 51 in the captured image 50 based on the user's setting. In this case, the post-processing unit 31B treats the area 51 as a detection area, and treats an area 53, which is the entire area of ​​the captured image 50 excluding the area 51, as a non-detection area.

[0052] The post-processing unit 31B advances the character string included in the detection area to subsequent processing and excludes the character string included in the non-detection area from the subsequent processing. In the example of Fig. 5, the post-processing unit 31B advances the character string ST14 to subsequent processing and excludes the character string ST13 from the subsequent processing.

[0053] Next, filtering of character strings based on a non-detection mask will be described. The user may set in advance an area in the captured image to which the non-detection mask is to be applied. In the example of FIG. 5 , a non-detection mask 55 is applied to an area 54 in the captured image 50 based on the user's setting. In this case, the post-processing unit 31B treats the area 54 as a non-detection area, and treats an area 56, which is the entire area of ​​the captured image 50 excluding the area 54, as a detection area. The post-processing unit 31B advances the character string ST13 included in the detection area to subsequent processing, and excludes the character string ST14 included in the non-detection area from the target of each subsequent processing.

[0054] Here, a supplementary explanation will be given regarding filtering based on character size in step St101 of the flowchart shown in FIG. 4 . When performing character recognition on a detected character string, the character recognition unit 31A detects a character rectangle for each character included in the character string. The character rectangle is a rectangle that encloses each character detected by the character recognition unit 31A and indicates the position, shape, and size of each character. In FIG. 5 , all character rectangles are approximately the same size, but the size of the character rectangle may vary depending on the size of the character. It is sufficient for the character rectangle to encompass at least the entire character. Whether the character rectangle encompasses a larger area than the character may vary depending on the function of the character recognition engine and user settings. Furthermore, in this embodiment, regardless of whether the character is tilted or not, the horizontal and vertical directions of the character rectangle are aligned with the horizontal and vertical directions of the captured image 50. By aligning the coordinate systems of the captured image 50 and the character rectangles, coordinate system conversion is unnecessary when dealing with the entire captured image 50 and when dealing with the character rectangles, thereby simplifying the configuration of the character recognition device 30. Referring to FIG. 5 , the character recognition unit 31A detects character rectangles CR1, CR2, CR3, CR4, CR5, and CR6 for each character in the character string ST14, for example, "123627." The character recognition unit 31A detects character rectangles and recognizes the characters within the character rectangles. The character recognition unit 31A also detects character string rectangles. The character string rectangle indicates the position, shape, and size of a character string. For example, the character recognition unit 31A determines that a set of multiple characters whose character rectangles overlap or touch each other is a single character string, and detects the character string rectangle. Furthermore, in this embodiment, regardless of whether the character string is tilted, the horizontal and vertical directions of the character string rectangle are aligned with the horizontal and vertical directions of the captured image 50. This eliminates the need for coordinate system conversion between when processing the entire captured image 50 and when processing character string rectangles, thereby simplifying the configuration of the character recognition device 30. The character recognition unit 31A can detect, for example, a rectangle that corresponds to the smallest and largest coordinates in the horizontal and vertical directions among the character rectangles that make up a character string, and thereby detect the rectangle as a character string rectangle.5, the character recognition unit 31A detects a character string rectangle SR1 of, for example, "123456" in the character string ST13. For convenience of explanation, the captured image 50 to which the detection mask or non-detection mask is applied is referred to, but the detection of character rectangles and character string rectangles by the character recognition unit 31A is performed before filtering based on the detection mask or non-detection mask, as described with reference to FIG.

[0055] Returning to the explanation of filtering of character strings based on a detection mask or a non-detection mask, if at least a portion of the character rectangle of a certain character is included in the non-detection area, the post-processing unit 31B excludes the character string containing that character from the target of each subsequent processing step.

[0056] For example, depending on the installation location of the imaging device 10, captured images may almost always contain character strings unrelated to trailer IDs, such as character strings written on signs or buildings, or the area in the captured image in which trailers appear may be largely limited. Therefore, by limiting the character strings to be subjected to various processes, such as output, using a detection mask or a non-detection mask, it is possible to omit various processes for character strings unrelated to trailer IDs. This, for example, reduces the processing load on the character recognition device 30 and improves the output accuracy of character strings that are considered to be trailer IDs.

[0057] In this way, the post-processing unit 31B performs horizontal combining processing on character strings that include only characters within a specified size range, among the character strings recognized by the character recognition unit 31A. The post-processing unit 31B also performs horizontal combining processing on character strings that are included in a specified area of ​​the captured image, among the character strings recognized by the character recognition unit 31A.

[0058] Next, the horizontal writing merging process in step St103 of the flowchart shown in FIG. 4 will be described with reference to FIGS. 6 and 7. For example, if a single character string, such as a trailer ID, contains a space in the captured image, the character recognition unit 31A may recognize the single character string as multiple character strings. Therefore, the post-processing unit 31B performs horizontal writing merging process based on specified conditions so that the single character string recognized as multiple character strings can be treated as a single character string in subsequent processing. FIG. 6 is a table diagram for explaining various merging process conditions according to the first embodiment. FIG. 7 is a schematic diagram for explaining the horizontal writing merging process according to the first embodiment.

[0059] 7 includes two character strings, ST15 and ST16. The character recognition unit 31A detects character string rectangles SR2 and SR3 for each of the two character strings.

[0060] In the following description, the horizontal direction in a captured image coincides with the x-axis direction, and the direction perpendicular to the horizontal direction in a captured image coincides with the y-axis direction. Furthermore, in the following description, the x-axis direction will be referred to as the horizontal and the y-axis direction as the vertical direction in a captured image. Furthermore, in the following description, the positive direction of the x-axis will sometimes be referred to as the right, the negative direction of the x-axis as the left, the positive direction of the y-axis as the up, and the negative direction of the y-axis as the down.

[0061] In order for two character strings to satisfy the horizontal writing combination condition, both of the two character strings must be written horizontally. In the captured image 60, both character strings ST15 and ST16 are written horizontally. Whether a character string is written horizontally or vertically is determined, for example, based on whether the line connecting the centers of the character rectangles of the first and last characters constituting the character string is aligned vertically or horizontally in the captured image. When the character string rectangles are arranged diagonally, for example, assuming that the horizontal direction is 0 degrees, the character string is determined to be written horizontally if the angle at which the character string rectangles are arranged is within a range of ±45 degrees, and is determined to be written vertically if outside this range. Note that other methods for determining whether a character string is written horizontally or vertically are known, and any method may be employed.

[0062] Furthermore, for two character strings to satisfy the horizontal writing combination condition, the value obtained by dividing the distance in the x-axis direction between the character string rectangles of the two character strings by the width of one character must be equal to or less than a predetermined threshold. The distance in the x-axis direction between two character string rectangles under the horizontal writing combination condition is the distance in the x-axis direction between the right end of the left character string rectangle and the left end of the right character string rectangle. This distance may be the distance projected in the x-axis direction of the line segment connecting the right end of the left character string rectangle and the left end of the right character string rectangle. In the captured image 60, the distance in the x-axis direction between character string rectangles SR2 and SR3 is distance L1. The width of one character may be, for example, the maximum or median of the x-axis lengths of the character rectangles of each character included in each of the two character strings. The maximum or median is used because the x-axis length of the character rectangles of some characters, such as "1" or "J," is short. Using an average value, for example, could result in the width of one character being too small due to the influence of characters with short x-axis lengths. The predetermined threshold may be arbitrarily set by the user, for example.

[0063] Furthermore, for two character strings to satisfy the horizontal writing combination condition, the value obtained by dividing the overlap of the character string rectangles of the two character strings in the y-axis direction by the height of the character string rectangles must be equal to or greater than a predetermined threshold. The overlap of two character string rectangles in the y-axis direction may be the overlap of the two character string rectangles in the y-axis direction when the two character string rectangles are projected in the y-axis direction. In the captured image 60, the overlap of character string rectangles SR2 and SR3 in the y-axis direction is overlap L2. The height of the character string rectangle may be the height of one of the two character string rectangles, in other words, the height of the character string rectangle with the shorter length in the y-axis direction. If the height of the taller character string rectangle is used, the value obtained by dividing the overlap of the character string rectangles in the y-axis direction by the height of the character string rectangle will be small if there is a difference in the sizes of the two character strings, which could result in an unnecessarily strict determination of the horizontal writing combination condition. In the captured image 60, the character string rectangle with the shorter height of the two character string rectangles is character string rectangle SR3. The height of character string rectangle SR3 is height L3. Therefore, the value obtained by dividing the overlap L2 by the height L3 is required to be equal to or greater than a predetermined threshold value. The predetermined threshold value may be arbitrarily set by the user, for example.

[0064] Furthermore, for two character strings to satisfy the horizontal combining condition, the left-hand string of the two character strings must contain four characters, and the right-hand string must contain six or seven characters. In the captured image 60, the left-hand string, character string ST15, contains four characters, and the right-hand string, character string ST16, contains seven characters. This number of characters is set based on a known format, such as a trailer ID. In this embodiment, it is assumed that the trailer ID is known to consist of four alphabetic characters and six or seven numeric characters. Spaces may be provided between the alphabetic characters and the numeric characters to make the trailer ID easier to read, or spaces may be created between the character detection frames due to differences in character types. Therefore, a trailer ID with such a format is likely to be detected as a four-character string and a six- or seven-character string, rather than as a single character string. Note that if the format of the trailer ID, etc., is different, different conditions may be set as the horizontal combining condition depending on the format. Furthermore, conditions other than the number of characters may be used as the horizontal combining condition. For example, if the target is a trailer ID having the format described above, it is possible to use as a horizontally written combining condition that the left-hand string consists only of English letters and the right-hand string consists only of numbers.

[0065] In the example of Figure 7, if the value obtained by dividing the distance L1 by the width of one character (in other words, the distance between character strings relative to the width of one character) is equal to or less than a predetermined threshold, and the value obtained by dividing the overlap L2 by the height L3 (in other words, the overlap between character strings relative to the height of one character) is equal to or greater than a predetermined threshold, then the character strings ST15 and ST16 satisfy the horizontally written combining condition. In this case, the post-processing unit 31B combines the character strings ST15 and ST16. Then, the post-processing unit 31B treats the combined character string "MRKU4945420" as a single character string.

[0066] The larger the character strings appear in the captured image, the greater the likelihood that the distance L1 will be. Therefore, in this embodiment, the influence of the character size in the captured image is eliminated by using the distance between character strings relative to the width of one character in the width direction. The threshold value used when making a judgment using the distance between character strings relative to the width of one character may be, for example, "1." When this threshold value is "1," it means that the horizontally combined condition is met if the distance L1 is equal to or less than the width of one character. Since it is rare to write a single character string with a space of one character or more between characters, using this threshold value makes it possible to restore the single character string that was originally intended to be written when it is mistakenly recognized as two character strings. The specific value of the threshold is merely an example, and values ​​smaller or larger than "1" may also be used.

[0067] Similarly, in this embodiment, the influence of the size of the characters in the captured image is eliminated by using the overlap of character strings relative to the height of one character in the height direction. Furthermore, the threshold value used when making a judgment based on the overlap of character strings relative to the height of one character may be, for example, "0.5." When this threshold value is "0.5," it means that the horizontally-written combining condition is met if the overlap of character strings is equal to or greater than half the height of one character. Note that the specific value of the threshold is merely an example, and values ​​smaller or larger than "0.5" may also be used.

[0068] By combining two character strings that satisfy the horizontal combining conditions described with reference to FIGS. 6 and 7, it is highly likely that a character string that is considered to be a trailer ID can be detected. However, some of the horizontal combining conditions described with reference to FIGS. 6 and 7 may be omitted, or new conditions may be added. For example, the condition regarding the number of characters contained in the character string may be omitted. The condition regarding the number of characters is specific to a specific format and is therefore too strict when the purpose is to also detect trailer IDs and other formats. In this case, the post-processing unit 31B may determine that two character strings satisfy the horizontal combining condition if the horizontal distance between the two character strings in the captured image relative to the width of the characters contained in the two character strings is less than a predetermined threshold, and the overlap between the two character strings in the direction perpendicular to the horizontal direction relative to the height of the characters contained in the two character strings is greater than a predetermined threshold. Note that these conditions regarding the horizontal distance and the overlapping in the direction perpendicular to the horizontal direction are conditions that are applicable to character strings in general, so if only these conditions are used, there is a risk that the horizontal combining process will be performed on character strings that are unrelated to trailer IDs and other formats. Therefore, if the format of the trailer ID and the like is known, making a decision that also takes into account format-related conditions such as the number of characters and character type can reduce the possibility of incorrectly processing horizontally combined text.

[0069] Next, the two-line merging process in step St105 of the flowchart shown in FIG. 4 will be described with reference to FIGS. 6, 8, and 9. For example, if a single character string, such as a trailer ID, is written across multiple lines in a captured image, the character recognition unit 31A may recognize each of the multiple lines as a separate character string. Therefore, the post-processing unit 31B performs a two-line merging process or a three-line merging process based on specified conditions to treat a single character string that is written across multiple lines and recognized as multiple character strings as a single character string in subsequent processing. For ease of explanation, the three-line merging process in step St104 of the flowchart shown in FIG. 4 will be described after the two-line merging process. FIGS. 8 and 9 are schematic diagrams for explaining the two-line merging process according to the first embodiment.

[0070] First, a case where the character strings are written horizontally will be described with reference to Fig. 8. The captured image 70 shown in Fig. 8 includes two horizontally written character strings ST17 and ST18. The character recognition unit 31A detects character string rectangles SR4 and SR5 for each of the two character strings.

[0071] For two horizontally written character strings to satisfy the conditions for the two-line merging process, in other words, the two-line merging conditions, the value obtained by dividing the y-axis distance between the character string rectangles of the two character strings by the height of one character must be equal to or less than a predetermined threshold. The y-axis distance between the two character string rectangles is the distance between the centers of the two character string rectangles. In the captured image 70, the y-axis distance between the character string rectangles SR4 and SR5 is distance L4. The character height may be the height of the character rectangles of each character contained in each of the two character strings, in other words, the maximum or median of the lengths in the y-axis direction. The maximum or median is used because the y-axis length of the character rectangles of some characters, such as "-" or "_", is short. If an average value or the like is used, the height of one character may be too small due to the influence of characters with a short y-axis length. The predetermined threshold may be arbitrarily set by the user, for example.

[0072] Furthermore, for two horizontally written character strings to satisfy the two-line joining condition, the value obtained by dividing the overlap of the character string rectangles of the two character strings in the x-axis direction by the width of the character string rectangles must be equal to or greater than a predetermined threshold. The overlap of the two character string rectangles in the x-axis direction may be the overlap of the two character string rectangles in the x-axis direction when the two character string rectangles are projected in the x-axis direction. In the captured image 70, the overlap of the character string rectangles SR4 and SR5 in the x-axis direction is overlap L5. The width of the character string rectangle may be the width of the two character string rectangles, in other words, the width of the character string rectangle with the shorter x-axis length. If the width of the character string rectangle with the longer x-axis length is used, the value obtained by dividing the overlap of the character string rectangles in the x-axis direction by the width of the character string rectangle will be small if there is a difference in the sizes of the two character strings, which may result in an unnecessarily strict determination of the two-line joining condition. In the captured image 70, the character string rectangle with the smaller width of the two character string rectangles is character string rectangle SR4. The width of character string rectangle SR4 is width L6. Therefore, the value obtained by dividing the overlap L5 by the width L6 is required to be equal to or greater than a predetermined threshold value. The predetermined threshold value may be arbitrarily set by the user, for example.

[0073] In the example of Figure 8, if the value obtained by dividing distance L4 by the height of one character (in other words, the distance between character strings relative to the height of one character) is equal to or less than a predetermined threshold, and the value obtained by dividing overlap L5 by width L6 (in other words, the overlap between character strings relative to the width of one character) is equal to or greater than a predetermined threshold, character strings ST17 and ST18 satisfy the two-line join condition. In this case, post-processing unit 31B joins character strings ST17 and ST18. Then, post-processing unit 31B treats the joined character string "JBHU24967545325" as a single character string.

[0074] In this way, the post-processing unit 31B may determine that the two horizontally written character strings divided into two lines satisfy the two-line joining condition if the distance in the horizontal and vertical directions in the captured image of the two character strings relative to the height of the characters contained in the two character strings is less than a predetermined threshold, and the horizontal overlap of the two character strings relative to the width of the characters contained in the two character strings is greater than a predetermined threshold.

[0075] Next, a case where character strings are written vertically will be described with reference to Fig. 9. The captured image 80 shown in Fig. 9 includes two vertically written character strings ST19 and ST20. The character recognition unit 31A detects character string rectangles SR6 and SR7 for each of the two character strings.

[0076] For two vertically written character strings to satisfy the two-line joining condition, the value obtained by dividing the distance in the x-axis direction between the character string rectangles of the two character strings by the width of one character must be equal to or less than a predetermined threshold. Unlike the case of the horizontally written joining condition, the distance in the x-axis direction between the two character string rectangles under the two-line joining condition is the distance in the x-axis direction between the centers of the two character string rectangles. In the captured image 70, the distance in the x-axis direction between the character string rectangles SR6 and SR7 is distance L7. The width of one character was described above in the explanation of the horizontally written joining process, so a description of it will be omitted here. The predetermined threshold may be set arbitrarily by the user, for example.

[0077] Furthermore, for two vertically written character strings to satisfy the two-line merging condition, the value obtained by dividing the overlap of the character string rectangles of the two character strings in the y-axis direction by the height of the character string rectangles must be equal to or greater than a predetermined threshold. The overlap of the two character string rectangles in the y-axis direction and the height of the character string rectangles have been described above in the horizontally writing merging process, and therefore will not be described here. In the captured image 70, the overlap of the character string rectangles SR6 and SR7 in the y-axis direction is L8. In the captured image 70, the height of the character string rectangle ST19, which has the shorter height of the two character strings, is L9. Therefore, the value obtained by dividing the overlap L8 by the height L9 must be equal to or greater than a predetermined threshold. The predetermined threshold may be set arbitrarily by the user, for example.

[0078] In the example of Figure 9, if the value obtained by dividing distance L7 by the width of one character (in other words, the distance between character strings relative to the width of one character) is equal to or less than a predetermined threshold, and the value obtained by dividing overlap L8 by height L9 (in other words, overlap L8 relative to height L9) is equal to or greater than a predetermined threshold, character strings ST19 and ST20 satisfy the two-line join condition. In this case, post-processing unit 31B joins character strings ST19 and ST20. Then, post-processing unit 31B treats the joined character string "JBHU2245678" as a single character string.

[0079] In this way, the post-processing unit 31B may determine that the two character strings satisfy the two-line joining condition if the horizontal distance in the captured image of the two character strings relative to the width of the characters contained in the two character strings that are written vertically and that are divided into two lines is less than a predetermined threshold, and if the overlap of the two character strings in the direction perpendicular to the horizontal direction relative to the height of the characters contained in the two character strings is greater than a predetermined threshold.

[0080] It is highly likely that a character string that is considered to be a trailer ID can be detected by combining two character strings that satisfy the two-line combining conditions described with reference to Figures 6, 8, and 9. However, some of the two-line combining conditions described with reference to Figures 6, 8, and 9 may be omitted, or new conditions may be added.

[0081] Next, the three-row joining process in step St104 of the flowchart shown in Fig. 4 will be described with reference to Fig. 6 and Fig. 10. Fig. 10 is a schematic diagram for explaining the three-row joining process according to the first embodiment.

[0082] 10 includes nine horizontally written character strings ST21, ST22, ST23, ST24, ST25, ST26, ST27, ST28, and ST29. The character recognition unit 31A detects a character string rectangle SR8, a character string rectangle SR9, a character string rectangle SR10, a character string rectangle SR11, a character string rectangle SR12, a character string rectangle SR13, a character string rectangle SR14, a character string rectangle SR15, and a character string rectangle SR16 for each of the nine character strings.

[0083] In order for three character strings to satisfy the conditions for the three-line joining process, in other words, the three character strings to satisfy the three-line joining conditions, the three character strings are required to be candidates for the three-line joining process. Therefore, first, an example of the process up to when the post-processing unit 31B selects three character strings from among multiple character strings as candidates for the three-line joining process will be described.

[0084] First, the post-processing unit 31B numbers the character string rectangles included in the captured image 90 in ascending order of the y coordinate of the center of the character string rectangle. As a result, the post-processing unit 31B numbers the character string rectangle SR8 as number 1, the character string rectangle SR9 as number 2, and so on. The post-processing unit 31B assigns numbers 1 to 9 to the character string rectangle SR8, character string rectangle SR9, character string rectangle SR10, character string rectangle SR11, character string rectangle SR12, character string rectangle SR13, character string rectangle SR14, character string rectangle SR15, and character string rectangle SR16, in that order.

[0085] The post-processing unit 31B sets a character string rectangle for the middle line (second line) of three character strings that can be candidates for the three-line joining process. For example, the post-processing unit 31B may first set the character string rectangle with the second largest y coordinate as the character string rectangle for the second line. That is, the post-processing unit 31B may set the character string rectangle that is highest in the y-axis direction in the captured image 70 among the character string rectangles for character strings that can be the second line of the three character strings as the second line. In the example of FIG. 10 , the post-processing unit 31B first sets the character string rectangle SR9 as the character string rectangle for the second line.

[0086] Next, the post-processing unit 31B searches for character string rectangles above and below the character string rectangle set as the second line. The character string rectangle above the character string rectangle SR9 is the character string rectangle SR8, and the character string rectangles below the character string rectangle SR9 are the character string rectangles SR10, SR11, SR12, SR13, SR14, SR15, and SR16. The post-processing unit 31B searches for the character string rectangle above the character string rectangle SR9 set as the second line that is closest to the character string rectangle SR9. The distance between the character string rectangles in the three-line joining process is the distance between the centers of the character string rectangles. The post-processing unit 31B also searches for the character string rectangle below the character string rectangle SR9 set as the second line that is closest to the character string rectangle SR9. Of the character string rectangles above character string rectangle SR9, the character string rectangle closest to character string rectangle SR9 is character string rectangle SR8. Furthermore, of the character string rectangles below character string rectangle SR9, the character string rectangle closest to character string rectangle SR9 is character string rectangle SR10. Post-processing unit 31B determines whether the character strings enclosed by character string rectangles SR8, SR9, and SR10, respectively, i.e., character strings ST21, ST22, and ST23, are candidates for the three-line joining process.

[0087] The post-processing unit 31B determines whether the top two character string rectangles of character string rectangles SR8, SR9, and SR10, i.e., character string rectangles SR8 and SR9, satisfy the two-line joining condition when the character string is written horizontally. The post-processing unit 31B also determines whether the bottom two character string rectangles of character string rectangles SR8, SR9, and SR10, i.e., character string rectangles SR9 and SR10, satisfy the two-line joining condition when the character string is written horizontally. Furthermore, the post-processing unit 31B determines whether the number of characters in character strings ST21, ST22, and ST23, respectively, from top to bottom, is four, three, and three. If the post-processing unit 31B determines that the pair of character string rectangles SR8 and SR9 and the pair of character string rectangles SR9 and SR10 each satisfy the two-line joining condition when the character strings are written horizontally, and that the number of characters in character strings ST21, ST22, and ST23, respectively, from top to bottom, are four, three, and three, the post-processing unit 31B selects the three character strings, character string ST21, character string ST22, and character string ST23, as candidates for the three-line joining process. If the three character strings are character string ST21, character string ST22, and character string ST23, for example, because character string ST21 has three characters, the post-processing unit 31B does not select these three character strings as candidates for the three-line joining process.

[0088] In this way, if the string rectangles of the top two lines of strings and the string rectangles of the bottom two lines of strings among three strings satisfy the conditions for two-line joining processing, and the number of characters in the three strings is four, three, and three, respectively, from top to bottom, the post-processing unit 31B designates the three strings as candidates for three-line joining processing.

[0089] Next, the post-processing unit 31B sets the character string rectangle of the second line of the three character strings that can be candidates for the three-line joining process as the character string rectangle SR10, which is one line below the character string rectangle SR9, and repeats the above-described process. Specifically, the post-processing unit 31B searches for character string rectangles above the character string rectangle SR10 and below the character string rectangle SR10. The post-processing unit 31B also determines whether three character strings, including at least the character string ST23 enclosed by the character string rectangle SR10, are candidates for the three-line joining process. In this way, the post-processing unit 31B sequentially sets the character string rectangles of the second line of the three character strings that can be candidates for the three-line joining process, and determines whether the three searched character strings are candidates for the three-line joining process.

[0090] Here, we will explain the case where the post-processing unit 31B sets a character string rectangle SR12 as the character string rectangle for the second line of three character strings that can be candidates for the three-line joining process. The post-processing unit 31B searches for character string rectangles above and below the character string rectangle SR12 set as the second line. The post-processing unit 31B then searches for the character string rectangle that is closest to the character string rectangle SR12 among the character string rectangles above the character string rectangle SR12, and the character string rectangle that is closest to the character string rectangle SR12 among the character string rectangles below the character string rectangle SR12.

[0091] Of the character string rectangles above SR12, the character string rectangle closest to SR12 is SR10. For example, in the y-axis direction, character string rectangle SR11 is closer to SR12 than character string rectangle SR10. However, the distance L11 between character string rectangle SR12 and character string rectangle SR10 is smaller than the distance L10 between character string rectangle SR12 and character string rectangle SR11.

[0092] Of the character string rectangles below SR12, the character string rectangle closest to SR12 is SR14. For example, in the y-axis direction, character string rectangle SR13 is closer to SR12 than character string rectangle SR14. However, the distance L13 between character string rectangle SR12 and character string rectangle SR14 is smaller than the distance L12 between character string rectangle SR12 and character string rectangle SR13.

[0093] The post-processing unit 31B determines whether the character strings enclosed by the character string rectangles SR10, SR12, and SR14, i.e., the character strings ST23, ST25, and ST27, respectively, are candidates for the three-line joining process.

[0094] The post-processing unit 31B determines whether the top two character string rectangles among the character string rectangles SR10, SR12, and SR14, i.e., the character string rectangles SR10 and SR12, satisfy the two-line joining condition when the character string is written horizontally. The post-processing unit 31B also determines whether the bottom two character string rectangles among the character string rectangles SR10, SR12, and SR14, i.e., the character string rectangles SR12 and SR14, satisfy the two-line joining condition when the character string is written horizontally. Furthermore, the post-processing unit 31B determines whether the number of characters in the character strings ST23, ST25, and ST27, respectively, from top to bottom, is four, three, and three. For convenience, detailed description will be omitted, but the post-processing unit 31B determines that the pair of the character string rectangles SR10 and SR12 and the pair of the character string rectangles SR12 and SR14 each satisfy the two-line joining condition when the character string is written horizontally. The number of characters in the character strings ST23, ST25, and ST27, from top to bottom, is four, three, and three, respectively. Therefore, the post-processing unit 31B determines that the character strings ST23, ST25, and ST27 are candidates for the three-line joining process. As a result, the character strings ST23, ST25, and ST27 are candidates for the three-line joining process.

[0095] Although not shown in the figure, there are cases where overlapping character string rectangles, i.e., character strings, become candidates for the three-line joining process. For example, in the example of FIG. 10 , three character strings, namely, character string ST23, character string ST25, and character string ST27, are candidates for the three-line joining process. If three character strings, including character string ST23, which are different from the three character strings ST23, ST25, and ST27, become candidates for the three-line joining process, character string ST23 becomes a candidate for the three-line joining process due to the overlap. In this way, when a character string overlaps and becomes a candidate for the three-line joining process, the combination with the smallest sum of the distances between the three character strings including the overlapping character string satisfies the three-line joining condition. The sum of the distances between the three character strings is, for example, the sum of distances L11 and L13, as explained with reference to FIG. 10 . Note that if overlapping character strings are not candidates for the three-line joining process, the candidate for the three-line joining process will simply satisfy the three-line joining condition.

[0096] 10, the post-processing unit 31B combines the character strings ST23, ST25, and ST27, which satisfy the three-line combining condition, and then treats the combined character string "HGUU123456" as a single character string.

[0097] A character string that is likely to be a trailer ID can be detected by combining three character strings that satisfy the three-line combining conditions described with reference to Figures 6 and 10. However, some of the three-line combining conditions described with reference to Figures 6 and 10 may be omitted, or new conditions may be added.

[0098] For example, the condition that the number of characters in the string is four, three, and three, from top to bottom, may be omitted or changed. This condition is difficult to set unless the number of digits in the trailer ID and the location where the line break occurs in the trailer ID are known. If the number of digits in the trailer ID is different or it is known that the line break occurs in a different location in the trailer ID, the number of digits in the string used as the three-line joining condition may be set based on that information. Furthermore, if the number of digits in the trailer ID or the location where the line break occurs is unknown or there are multiple possibilities, the condition regarding the number of digits in the string may be omitted. Another example of a new condition is a condition using the type of characters contained in the string. For example, if it is known that the trailer ID starts with an alphabetic character and ends with a number, the three-line joining condition may be that the first string from the top contains an alphabetic character, the second or third string contains a number, and the third string ends with a number.

[0099] As described with reference to FIG. 4 , the post-processing unit 31B outputs character strings recognized by the character recognition unit 31A that were not filtered out in the filtering processes of steps St101 and St102. The output character strings include character strings combined by the horizontally combining process, the two-line combining process, or the three-line combining process. Furthermore, if the character string to be output has already been output, the post-processing unit 31B prohibits the output of a character string identical to the previously output character string until a specified time has elapsed since the previous output by an output restriction process.

[0100] The post-processing unit 31B can output a character string that is considered to be a trailer ID by combining multiple character strings using a horizontal combining process, a two-line combining process, or a three-line combining process. Furthermore, if a character string that is considered to be a trailer ID is included in the captured image without being divided into multiple character strings, the post-processing unit 31B can output the character string that is considered to be a trailer ID regardless of the results of the various combining processes, provided that the character string that is considered to be a trailer ID is not excluded by the various filtering processes.

[0101] (Modification) In the above-described first embodiment, an example was shown in which the post-processing unit 31B of the character recognition device 30 outputs a character string that is considered to be a trailer ID in order to detect a character string that is considered to be a trailer ID from among the character strings included in the captured image. However, the character strings that the post-processing unit 31B can output are not limited to character strings that are considered to be trailer IDs. The post-processing unit 31B may also perform various filtering and various combining processes on character strings that are considered to be desired IDs other than trailer IDs, and output the resulting strings. This makes it possible to detect a character string that is considered to be a desired ID from among the character strings included in the captured image. The conditions for the various combining processes described with reference to FIG. 6 and the like may be set according to the desired ID or character string to be detected.

[0102] In the first embodiment, the post-processing unit 31B performs the horizontal writing merge process, the three-line merge process, and the two-line merge process in this order (see FIG. 4). However, this is not limiting, and the post-processing unit 31B may change the order of the merge processes or omit one or more of the merge processes. For example, the post-processing unit 31B may perform the two-line merge process after the horizontal writing merge process, and may omit the three-line merge process.

[0103] In the first embodiment, an example has been described in which the three-line join process and the two-line join process are not performed when the horizontally-written characters join process is performed. However, even when the horizontally-written characters join process is performed, the three-line join process or the two-line join process may be further performed. In this way, even if, for example, one of the character strings to be joined by the two-line join process or the three-line join process is erroneously detected as a plurality of character strings divided horizontally, the accurate trailer ID, etc. can be restored by the horizontally-written characters join process and the two-line join process or the three-line join process.

[0104] In the first embodiment, only the two-line joining process was described for the case where the character string is written vertically. However, the concepts of the horizontally written character joining process and the three-line joining process may also be applied to the case where the character string is written vertically. The conditions and processes for the case where the character string is written vertically can be realized by switching the horizontal and vertical directions of the conditions and processes described in the first embodiment, so a detailed description thereof will be omitted.

[0105] (Summary of First Embodiment) The following technology is disclosed by the above description of the first embodiment. Note that, in parentheses, examples of components corresponding to the first embodiment are shown, but the present invention is not limited to these.

[0106] (Technology 1) A character recognition device (e.g., character recognition device 30) includes a recognition unit (e.g., character recognition unit 31A) that recognizes character strings from an captured image (e.g., captured image 60) that includes a trailer (e.g., trailer TR) captured by an imaging device (e.g., imaging device 10), and a post-processing unit (e.g., post-processing unit 31B) that performs horizontal combining processing to combine two horizontally written character strings (e.g., character string ST15 and character string ST16) that are aligned horizontally in the captured image, among the character strings recognized by the recognition unit, that satisfy a horizontal combining condition for combining the two horizontally written character strings, and outputs recognized character strings that include the combined character strings.

[0107] This allows the recognition unit of the character recognition device to recognize character strings from a captured image that includes a trailer. Furthermore, the post-processing unit of the character recognition device can combine, among the character strings recognized by the recognition unit, character strings that have been recognized as multiple character strings due to the inclusion of spaces, etc., so that the combined character string can be processed as a single character string. Furthermore, the post-processing unit can output the character string recognized by the recognition unit, including the combined character string. The output character string is then determined, for example, by a character string determination device, to determine whether it is a trailer ID. This allows the character recognition device to detect, from among multiple character strings, a character string that is considered to be a desired ID related to trailer identification.

[0108] (Technology 2) In the character recognition device described in Technology 1, the post-processing unit determines that two character strings satisfy the horizontal combination condition if the horizontal distance in the captured image of the two character strings relative to the width of the characters contained in the two character strings is equal to or less than a predetermined threshold, and if the overlap of the two character strings in a direction perpendicular to the horizontal direction relative to the height of the characters contained in the two character strings is equal to or greater than a predetermined threshold.

[0109] This allows the character recognition device to combine two horizontally written character strings, for example, when the distance between the two character strings is small and the positions in the height direction (direction perpendicular to the horizontal direction) are approximately the same.

[0110] (Technology 3) In the character recognition device described in Technology 1 or 2, the post-processing unit performs a three-line joining process to join three character strings that satisfy a three-line joining condition for joining character strings that are divided into three lines, among the character strings recognized by the recognition unit.

[0111] As a result, when one character string is divided into three lines and recognized as three character strings, the character recognition device can combine the three character strings and output them as one character string.

[0112] (Technology 4) In the character recognition device described in Technology 3, the post-processing unit performs a two-line joining process to join two character strings that have been recognized by the recognition unit and for which a three-line joining process has not been performed and that satisfy a two-line joining condition for joining character strings that are split into two lines.

[0113] As a result, when one character string is split into two lines and recognized as two character strings, the character recognition device can combine the two character strings and output them as a single character string.

[0114] (Technology 5) In the character recognition device described in Technology 4, the character string divided into three lines is a character string written horizontally in the captured image, and the post-processing unit determines that the character string divided into three lines satisfies the three-line joining condition if the top two character strings in a direction perpendicular to the horizontal direction of the character string divided into three lines satisfy the two-line joining condition and the bottom two character strings in a vertical direction satisfy the two-line joining condition.

[0115] This allows the character recognition device to combine character strings by prioritizing the three-line combining process over the two-line combining process.

[0116] (Technology 6) In the character recognition device described in Technology 1 or 2, the post-processing unit performs a two-line joining process to join two character strings that satisfy a two-line joining condition for joining character strings that are split into two lines, among the character strings recognized by the recognition unit.

[0117] As a result, when one character string is split into two lines and recognized as two character strings, the character recognition device can combine the two character strings and output them as a single character string.

[0118] (Technology 7) In the character recognition device described in any one of Technologies 4 to 6, the character string divided into two lines is a character string written horizontally in the captured image, and the post-processing unit determines that the two character strings satisfy the two-line joining condition if the distance between the two character strings in the direction perpendicular to the horizontal direction with respect to the height of the characters contained in the two character strings is equal to or less than a predetermined threshold, and if the overlap between the two character strings in the horizontal direction with respect to the width of the characters contained in the two character strings is equal to or greater than a predetermined threshold.

[0119] This allows the character recognition device to combine two character strings written horizontally on two lines, for example, when the vertical distance (direction perpendicular to the horizontal direction) between the two character strings is small and there is a certain amount of overlap when projected horizontally.

[0120] (Technology 8) In the character recognition device described in any one of Technologies 4 to 6, the character string divided into two lines is a character string written vertically in the captured image, and the post-processing unit determines that the two character strings satisfy the two-line joining condition if the horizontal distance between the two character strings relative to the width of the characters contained in the two character strings is equal to or less than a predetermined threshold, and if the overlap between the two character strings in a direction perpendicular to the horizontal direction relative to the height of the characters contained in the two character strings is equal to or greater than a predetermined threshold.

[0121] This allows the character recognition device to combine two character strings written vertically on two lines, for example, when the horizontal distance between the two character strings is small and there is a certain amount of overlap when projected in the height direction (the direction perpendicular to the horizontal direction).

[0122] (Technology 9) In the character recognition device according to any one of technologies 1 to 7, the post-processing unit performs horizontal joining processing on a character string that includes only characters whose sizes are within a specified range, among the character strings recognized by the recognition unit.

[0123] This allows the character recognition device to exclude character strings that include characters that are extremely large or small in size, for example.

[0124] (Technology 10) In the character recognition device according to any one of technologies 1 to 8, the post-processing unit performs horizontal joining processing on character strings that are included in a specified area of ​​the captured image, among the character strings recognized by the recognition unit.

[0125] This allows the character recognition device to exclude character strings included in specific areas of the captured image. In other words, the character recognition device can perform various processes, such as output, on character strings included in specific areas of the captured image.

[0126] (Technology 11) In the character recognition device according to any one of technologies 1 to 9, when the post-processing unit outputs a recognized character string, it prohibits output of a character string identical to the recognized character string until a specified time has elapsed after the output.

[0127] This allows the character recognition device to prevent the repeated output of the same character string that is repeatedly recognized from the captured image.

[0128] (Technology 12) A character recognition method recognizes character strings from a captured image including a trailer captured by an imaging device, performs horizontal combining processing to combine two recognized character strings that satisfy a horizontal combining condition for combining two character strings that are aligned horizontally in the captured image, and outputs recognized character strings including the combined character string.

[0129] As a result, the character recognition method can obtain the same effect as Technique 1.

[0130] (Technology 13) A character recognition system (for example, character recognition system 1) includes at least one imaging device, a recognition unit that recognizes character strings from an imaged image including a trailer captured by the imaging device, and a post-processing unit that performs horizontal joining processing to join two character strings recognized by the recognition unit that satisfy a horizontal joining condition for joining two character strings that are aligned horizontally in the captured image, and outputs recognized character strings including the joined character string.

[0131] This allows the character recognition system to obtain the same effect as that of Technique 1.

[0132] Although the embodiments have been described above with reference to the accompanying drawings, the present disclosure is not limited to such examples. It is clear that a person skilled in the art can conceive of various modifications, alterations, substitutions, additions, deletions, and equivalents within the scope of the claims, and it is understood that these also fall within the technical scope of the present disclosure. Furthermore, the components in the above-described embodiments may be combined in any manner without departing from the spirit of the invention.

[0133] The disclosures of the specification, drawings and abstracts contained in the U.S. provisional application No. 63 / 575030 filed on April 5, 2024, and the Japanese patent application No. 2024-179287 filed on October 11, 2024, are incorporated herein by reference in their entirety.

[0134] The techniques of the present disclosure are useful as character recognition devices, character recognition methods, and character recognition systems.

[0135] TR Trailer LE Tractor ST1, ST2, ST3, ST4, ST5, ST6, ST7, ST8, ST9, ST10, ST11, ST12, ST13, ST14, ST15, ST16, ST17, ST18, ST19, ST20, ST21, ST22, ST23, ST24, ST25, ST26, ST27, ST28, ST29 Character string CR1, CR2, CR3, CR4, CR5, CR6 Character rectangle SR1, SR2, SR3, SR4, SR5, SR6, SR7, SR8, SR9, SR10, SR11, SR12, SR13, SR14, SR15, SR16 Character string rectangle 1 Character recognition system 10 Imaging device 20 Display device 30 Character recognition device 31 Processor 31A Character recognition unit 31B Post-processing unit 32 Memory 33 Communication interface 50, 60, 70, 80, 90 Captured image 51, 53, 54, 56 Area 52 Detection mask 55 Non-detection mask

Claims

1. A character recognition device comprising: a recognition unit that recognizes character strings from an image captured by an imaging device, including a trailer; and a post-processing unit that performs horizontal joining processing to join two character strings recognized by the recognition unit that satisfy horizontal joining conditions for joining two horizontally written character strings that are aligned horizontally in the captured image, and outputs the recognized character strings including the joined character string.

2. The character recognition device according to claim 1, wherein the post-processing unit determines that the two character strings satisfy the horizontal writing combination condition if the horizontal distance in the captured image of the two character strings relative to the width of the characters contained in the two character strings is equal to or less than a predetermined threshold, and if the overlap of the two character strings in a direction perpendicular to the horizontal direction relative to the height of the characters contained in the two character strings is equal to or greater than a predetermined threshold.

3. The character recognition device according to claim 1, wherein the post-processing unit performs a three-line joining process to join three character strings that satisfy a three-line joining condition for joining character strings that are split into three lines, among the character strings recognized by the recognition unit.

4. The character recognition device according to claim 3, wherein the post-processing unit performs a two-line joining process to join two character strings that have been recognized by the recognition unit and for which the three-line joining process has not been performed and that satisfy the two-line joining condition for joining character strings that are split into two lines.

5. The character recognition device of claim 4, wherein the character string divided into three lines is a character string written horizontally in the captured image, and the post-processing unit determines that the character string divided into three lines satisfies the three-line joining condition if the top two character strings in a direction perpendicular to the horizontal direction of the character string divided into three lines satisfy the two-line joining condition and the bottom two character strings in the vertical direction satisfy the two-line joining condition.

6. The character recognition device according to claim 1, wherein the post-processing unit performs a two-line joining process to join two character strings recognized by the recognition unit that satisfy a two-line joining condition for joining character strings split into two lines.

7. The character recognition device according to claim 5 or 6, wherein the character strings divided into two lines are character strings written horizontally in the captured image, and the post-processing unit determines that the two character strings satisfy the two-line joining condition if the distance between the two character strings in a direction perpendicular to the horizontal direction and the height of the characters contained in the two character strings is equal to or less than a predetermined threshold, and the overlap between the two character strings in the horizontal direction and the width of the characters contained in the two character strings is equal to or greater than a predetermined threshold.

8. The character recognition device according to claim 5 or 6, wherein the character strings divided into two lines are character strings written vertically in the captured image, and the post-processing unit determines that the two character strings satisfy the two-line joining condition if the horizontal distance between the two character strings relative to the width of the characters contained in the two character strings is equal to or less than a predetermined threshold, and the overlap of the two character strings in a direction perpendicular to the horizontal direction relative to the height of the characters contained in the two character strings is equal to or greater than a predetermined threshold.

9. The character recognition device according to claim 1, wherein the post-processing unit performs the horizontal joining process on character strings recognized by the recognition unit that include only characters whose sizes are within a specified range.

10. The character recognition device according to claim 1, wherein the post-processing unit performs the horizontal joining process on character strings recognized by the recognition unit that are included in a specified area of ​​the captured image.

11. The character recognition device according to claim 1, wherein, when the post-processing unit outputs the recognized character string, it prohibits output of a character string identical to the recognized character string until a specified time has elapsed after the output.

12. A character recognition method comprising: recognizing character strings from a captured image including a trailer captured by an imaging device; performing horizontal joining processing to join two recognized character strings that satisfy horizontal joining conditions for joining two character strings that are aligned horizontally in the captured image; and outputting the recognized character strings including the joined character string.

13. A character recognition system comprising: at least one imaging device; a recognition unit that recognizes character strings from an image captured by the imaging device, including a trailer; and a post-processing unit that performs horizontal writing combination processing to combine two character strings recognized by the recognition unit that satisfy horizontal writing combination conditions for combining two character strings that are aligned horizontally in the captured image, and outputs the recognized character strings including the combined character string.

Images