Recognition device, recognition method, and recognition program

Abstract

This recognition device acquires a time-series image acquired in an environment in which a vehicle is travelling, detects each of the characters of a predetermined character string from the image, evaluates a relationship between each of the detected characters of the character string, and recognizes the shape of an object containing the character string.

Description

[Technical Field]

[0001] The disclosed technology relates to a recognition device, a recognition method, and a recognition program. [Background technology]

[0002] There is a technology for recognizing the shape of other objects captured in images captured by an in-vehicle camera such as a drive recorder.

[0003] Here, assume that a problematic vehicle that repeatedly engages in dangerous behavior, such as tailgating, is observed in relation to other objects. Technology has been developed to automatically detect the problematic vehicle when it enters a detection area (see Non-Patent Document 1). It is expected that the problematic vehicle will repeatedly engage in dangerous behavior at different locations, different times, and toward other vehicles. Therefore, it is highly necessary to notify other vehicles that the vehicle is dangerous. To do this, it is important to obtain information that can identify the vehicle. However, it is unlikely that the vehicle repeatedly engaging in dangerous behavior will provide information that can be used to identify it, such as location information, behavioral history, or footage of the interior of the vehicle, so it must be identified externally.

[0004] To identify a vehicle, for example, the vehicle model, vehicle color, and the number printed on the vehicle registration plate (license plate) are useful information for identification. Figure 1 is a diagram that schematically illustrates the case where another vehicle is observed from an observation vehicle. As shown in Figure 1, it is assumed that another vehicle (A2) is observed from the observation vehicle (A1) and the shape of the license plate (A3) of the other vehicle is recognized as the target. Furthermore, there is a possibility that a vehicle of the same model or color is present in the vicinity of the vehicle in question, and identification of the license plate number is essential to uniquely identify the vehicle in question. [Prior art documents] [Non-patent literature]

[0005] [Non-Patent Document 1] Kenwood DRV-MR8500 dashcam equipped with AI sensing to automatically detect aggressive driving. URL: https: / / car.watch.impress.co.jp / docs / news / 1266971.html [Non-patent document 2] "License Plate Recognition", URL: "https: / / www.jstage.jst.go.jp / article / isciesci / 43 / 6 / 43_KJ00003974529 / _article / -char / ja / " [Non-patent document 3] "Real-Time Brazilian License Plate Detection and Recognition Using Deep Convolutional Neural Networks",URL:"http: / / www.inf.ufrgs.br / ~smsilva / real-time-brazilian-alpr / " [Non-patent document 4] "Ministry of Land, Infrastructure, Transport and Tourism / Local Edition License Plates", URL: "https: / / www.mlit.go.jp / jidosha / jidosha_tk6_000036.html" Summary of the Invention [Problem to be solved by the invention]

[0006] As an existing technology, a method has been proposed in which license plates are detected using object detection technology and the characters written on them are read (see Non-Patent Documents 2 and 3). This method involves a step-by-step process of front / rear detection, license plate detection, and character recognition (character detection).

[0007] Figure 2 shows the general flow of processing from license plate detection to recognition. The process is as follows: (1) video (time-series image) acquisition, (2) license plate detection, (3) region segmentation, and (4) character recognition for each region (OCR or object detection). License plate detection involves, for example, object detection, or contour extraction through binarization and detection from a rectangular shape. In some cases, detection accuracy can be improved by performing vehicle object detection before (2). In other cases, tilt correction can be performed before (3) so that the region shape becomes rectangular.

[0008] In the case of license plates in English-speaking countries, as assumed in Non-Patent Document 3, the plates consist of 26 alphabetic characters and 10 numbers, and the main character string is often large and written on a single line, so character recognition is expected to be relatively easy. However, in countries with localized license plate formats, such as Japan or China, it is necessary to recognize characters including hiragana and kanji characters divided into multiple lines, and it is expected that the patterns will become complex. Furthermore, the example in Non-Patent Document 3 performs three-stage object detection, which requires high computational costs.

[0009] For example, some characters are small, kanji characters are included, some digits of one to three digits are alphabetic, one hiragana character is included, and dots (·) are included. An example of a dot-included number is "·1-43," which is a number of three digits or less. There are also multiple types of license plate backgrounds depending on whether the vehicle is for general use or business use, a standard vehicle or a light passenger vehicle, or an expired vehicle inspection, and these are determined regardless of the vehicle color. Furthermore, the variety of backgrounds is increasing due to the influence of regional license plates (see Non-Patent Document 4). Thus, there are challenges with license plate recognition.

[0010] Furthermore, unlike expensive, high-performance cameras, vehicle-mounted cameras are significantly affected by subject blur, i.e., blur caused by the relative speed between the observation vehicle and the target vehicle. Since the shooting conditions, mainly exposure and reflection, vary each time, there are cases where recognition is difficult using conventional methods.

[0011] For example, since there is no relationship between the color of a vehicle and the license plate, if the license plate and the vehicle color are similar, the boundary becomes unclear, making object detection and shape recognition difficult. For example, a white license plate on a white or silver vehicle, or a black license plate on a black vehicle, etc. may be expected. In addition, even if the area of ​​the license plate is enlarged, there are cases where the parts other than the "XX-XX" numbers are unclear. This presents an issue regarding blurring of the subject.

[0012] The disclosed technology has been made in consideration of the above circumstances, and aims to provide a recognition device, a recognition method, and a recognition program that can evaluate detected characters and recognize an object even when it is difficult to recognize a specific object. [Means for solving the problem]

[0013] A first aspect of the present disclosure is a recognition device that includes an acquisition unit that acquires a time series of images captured in an environment in which a vehicle is traveling, a detection unit that detects each character of a predetermined string from the images, and a shape recognition unit that evaluates the relationship between each of the detected characters of the string and recognizes the shape of an object that includes the string.

[0014] A second aspect of the present disclosure is a recognition method that causes a computer to perform the following processes: acquire a time series of images captured in an environment in which a vehicle is traveling, detect each character of a predetermined string from the images, evaluate the relationship between each of the detected characters of the string, and recognize the shape of an object that includes the string.

[0015] A third aspect of the present disclosure is a recognition program that causes a computer to perform the following process: acquire a time series of images captured in an environment in which a vehicle is traveling, detect each character of a predetermined string from the images, evaluate the relationship between each of the detected characters of the string, and recognize the shape of an object that includes the string. [Effects of the Invention]

[0016] According to the disclosed technology, even when it is difficult to recognize a specific object, the detected characters can be evaluated and the object can be recognized. [Brief explanation of the drawings]

[0017] [Figure 1] FIG. 10 is a diagram schematically illustrating a case where another vehicle is observed from an observation vehicle. [Figure 2] FIG. 1 is a diagram showing a general flow of processing from detection to recognition of a license plate. [Figure 3] FIG. 10 is a diagram showing an example of a flow of recognizing a license plate according to the present embodiment. [Figure 4] FIG. 10 is a diagram showing an example of an image relating to conditions for selecting a character string. [Figure 5] FIG. 1 is a diagram illustrating the relationship between how a camera and an object are captured. [Figure 6] FIG. 2 is a block diagram showing a hardware configuration of the recognition device. [Figure 7] FIG. 2 is a block diagram showing the functional configuration of the recognition device of the present embodiment. [Figure 8] FIG. 10 is a diagram illustrating an example of detection using a detection rectangle from an image. [Figure 9] FIG. 10 is a diagram illustrating an example of coordinate information of a detected rectangle. [Figure 10] 10 is a flowchart showing the flow of a recognition process performed by the recognition device. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of the disclosed technology will be described below with reference to the drawings. Note that the same reference numerals are used to designate identical or equivalent components and parts in each drawing. Also, the dimensional proportions in the drawings are exaggerated for the sake of explanation and may differ from the actual proportions.

[0019] First, an overview of the present disclosure will be described. In the example described below, a case where the shape of a license plate of a target vehicle captured from an observation vehicle is described. The method of this embodiment does not detect a pattern or shape corresponding to a license plate, but rather recognizes a pattern of characters corresponding to a license plate as a number, and an area containing this character pattern as a license plate. Hereinafter, descriptions of character strings are assumed to represent character string patterns. This is because, regardless of the color and pattern of the background of a license plate, the character strings depicted conform to the rules for describing license plates. Hereinafter, the character string portion of a license plate may be referred to as a "number," and the license plate itself as an object may be referred to as a "plate." The method of this embodiment can be applied not only to license plates, but also to signs and other features with characters.

[0020] FIG. 3 is a diagram showing an example of the flow of license plate recognition according to this embodiment. For example, in the general example described in the problem section, license plates are detected. However, in the method of this embodiment, (1) video (time-series images) is acquired, and (2) character detection is performed to select characters corresponding to the license plate. Character selection is performed by, for example, selecting a character string on a license plate based on conditions such as four numbers spaced at a specified interval, being of approximately uniform height, and being located below the vanishing point of the image. Furthermore, (3) rather than segmenting the area, the area is enlarged. Then, (4) the adjacent area is combined with the results of character recognition. This allows a boundary line to be assumed even when the license plate and vehicle body colors are similar and the boundary is unclear. Furthermore, even when the background is unique, such as a license plate unique to a region, additional learning is not required.

[0021] In the following description of the embodiment, the explanation will focus on the "XX-XX" character string in the license plate number section. Considering the font's special characteristics and writing rules, it is assumed that the license plate can be recognized by the "XX-XX" portion alone. License plates are subject to restrictions, such as the installation position, angle, and placement within the plate. In this embodiment, the definition of characters in the character string includes numbers, symbols, hiragana, and kanji.

[0022] FIG. 4 shows examples of image conditions for selecting character strings. For example, the area below the vanishing point may be limited, or character strings may be detected from the lower half of the image. Furthermore, since the "XX-XX" portion contains only digits 0 through 9 and dots, if an in-vehicle camera captures the front or rear of a vehicle, the character strings will appear horizontally at specific intervals. Furthermore, a string of multiple English letters can be determined not to be a license plate number. Furthermore, if a special font is used, the font can also be used to narrow down the search. Furthermore, vertically written character strings, such as those found on utility pole advertisements, may be excluded from detection.

[0023] The existing technology utilized in this embodiment will be described.

[0024] To detect each character in a string, we utilize the object detection technology described in Reference 1. This technology detects, for example, objects and each alphanumeric character from an image and outputs coordinate information on the circumscribing rectangular image (the XY coordinates of the upper left and the XY coordinates of the lower right of the rectangle), making it possible to detect each character in a string. [Reference 1] "YOLO: Real-Time Object Detection", URL: "https: / / pjreddie.com / darknet / yolo / "

[0025] It also utilizes the camera principle described in Reference 2. [Reference 2] "Perspective", URL:" http: / / www.persfreaks.jp / main / intro / pers / "

[0026] When using a monocular camera such as a typical in-vehicle camera, other vehicles or buildings are drawn so that they converge to a vanishing point. Furthermore, if an image is used that is less susceptible to lens distortion, such as when using a non-wide-angle lens or when cropping out an area with less distortion from a wide-angle lens, the size of objects in the image can be approximated like a perspective drawing, and changes according to a law with respect to the distance from a reference point.

[0027] Figure 5 is a diagram that explains the relationship between the camera and how an object is captured. If the camera is capturing a straight shot of the front or rear of another vehicle, the surface parallel to the projection plane, i.e., the front or rear of the vehicle body, will appear in the image at the same size if the distance in the depth direction is the same. This is because the camera and the projection plane can be expressed as similar triangles, and the length of the base and the height will be the same value. In other words, no matter how many lanes are between the onboard camera installed on the observation vehicle and the other vehicle whose license plate number you want to read, the characters on the license plate itself will appear the same.

[0028] Based on the above, the configuration of this embodiment will be described.

[0029] FIG. 6 is a block diagram showing the hardware configuration of the recognition device 100. As shown in FIG.

[0030] 6, the recognition device 100 includes a CPU (Central Processing Unit) 11, a ROM (Read Only Memory) 12, a RAM (Random Access Memory) 13, a storage 14, an input unit 15, a display unit 16, and a communication interface (I / F) 17. Each component is connected to each other via a bus 19 so as to be able to communicate with each other.

[0031] The CPU 11 is a central processing unit that executes various programs and controls each part. That is, the CPU 11 reads a program from the ROM 12 or the storage 14 and executes the program using the RAM 13 as a work area. The CPU 11 controls each of the above components and performs various arithmetic processing in accordance with the program stored in the ROM 12 or the storage 14. In this embodiment, the ROM 12 or the storage 14 stores a recognition program.

[0032] The ROM 12 stores various programs and various data. The RAM 13 temporarily stores programs or data as a working area. The storage 14 is configured with a storage device such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive) and stores various programs including the operating system and various data.

[0033] The input unit 15 includes a pointing device such as a mouse and a keyboard, and is used to perform various inputs.

[0034] The display unit 16 is, for example, a liquid crystal display, and displays various information. The display unit 16 may function as the input unit 15 by adopting a touch panel system.

[0035] The communication interface 17 is an interface for communicating with other devices such as terminals, etc. For this communication, for example, a wired communication standard such as Ethernet (registered trademark) or FDDI, or a wireless communication standard such as 4G, 5G, or Wi-Fi (registered trademark) is used.

[0036] Next, a description will be given of each functional configuration of the recognition device 100. Fig. 7 is a block diagram showing the functional configuration of the recognition device of this embodiment. Each functional configuration is realized by the CPU 11 reading out a recognition program stored in the ROM 12 or storage 14, expanding it in the RAM 13, and executing it.

[0037] As shown in FIG. 7, the recognition device 100 includes an acquisition unit 110, a recognition unit 112, and a storage unit 114.

[0038] The acquisition unit 110 acquires time-series images from the video captured by the on-board camera of the observation vehicle.

[0039] The recognition unit 112 includes a detection unit 120 and a shape recognition unit 122 .

[0040] Correspondence information relating to target features is stored in the storage unit 114. The correspondence information is, for example, information identifying the type of font on the license plate or sign, and information on restrictions such as the spacing between numbers.

[0041] The detection unit 120 detects each character of a string of characters from an image. As shown in Fig. 8, using object detection technology, characters represented by numbers (or symbol dots, etc.) in an image are detected using a detection rectangle that represents the range of the character defined by coordinates. As shown in Fig. 9, the position of a character in an image can be detected as coordinates, and a detection rectangle is obtained using the coordinates of each point: top left X, top left Y, bottom right X, and bottom right Y.

[0042] Furthermore, the detection unit 120 may detect each character in a string of characters in a specific font using a model that has been trained in advance to detect characters in a specific font.

[0043] The shape recognition unit 122 evaluates the relationship between each of the characters in the detected character string and recognizes the shape of the object containing the character string.

[0044] An example of evaluation by the shape recognition unit 122 will be described. To evaluate the relationship, the shape recognition unit 122 evaluates the positional relationship of each detection rectangle corresponding to each character and evaluates the pixel relationship of the detection rectangle. The example shown in FIG. 8 will be described. Regarding the positional relationship, whether each of the characters corresponding to the license plate number is recognized is evaluated. The degree of proximity is evaluated along the horizontal axis. 5 and 6, 7 and 8, and 3 and 0 are close to each other along the horizontal axis of each image. For the four characters, the degree of separation between the second and third characters is evaluated. 6 and 7 are slightly farther apart than the distance between 5 and 6, and 7 and 8 along the horizontal axis. The vertical position coordinates of 5, 6, 7, and 8, and the coordinates of 3 and 0 are adjacent. Therefore, the degree of proximity is evaluated for the four characters. Regarding the pixel relationship, the color distribution within each detection rectangle is evaluated, and it is evaluated whether 5, 6, 7, and 8 are the same color and gray, and whether 3 and 0 are the same color and a color similar to blue or navy blue. In the above evaluation example, the conditions for recognizing a number are that the spacing between the first and second characters, and the third and fourth characters are close to each other on the horizontal axis, the separation / spacing between the second and third characters of the string meets the constraints, and the vertical position coordinates are adjacent.The conditions for recognizing a number are that the pixels in the detection rectangle for the four characters have a similar color distribution.Recognition may also be performed in units of two or four characters.If it is performed in units of two characters, it is evaluated whether numbers 5 and 6, and numbers 7 and 8 are close to each other on the horizontal axis.Hyphens between numbers may also be taken into consideration.

[0045] The shape recognition unit 122 also determines whether the fonts of the characters in the string are the same, and evaluates the positional relationship between fonts determined to be the same. The object may be identified from a list of features for which fonts are known to be used, referencing the correspondence information in the storage unit 114. The font may also be determined by comparing each detected rectangle using features such as the aspect ratio of each character.

[0046] With the above steps, the shape recognition unit 122 can recognize the "XX-XX" portion, which corresponds to the license plate. Therefore, the shape recognition unit 122 may additionally recognize characters surrounding the "XX-XX" portion, which corresponds to the license plate. The shape of the target may also be recognized and detected based on the size of the characters. For example, when attempting to recognize the shape of a license plate, the shape recognition unit 122 may consider the presence of additional characters above and to the left of "XX-XX," the character size of each character in "XX-XX," and the number of pixels in the vertical and horizontal directions of the detection rectangle. Then, the shape recognition unit 122 may search for surrounding pixels of the same color as the characters "XX-XX" as characters or small elements that cannot be recognized as characters, and infer pixels other than the number on the license plate.

[0047] The shape recognition unit 122 may also perform additional processing, such as searching the area surrounding the four-character string corresponding to the license plate number, and determining that the area is a license plate area if there are any surrounding pixels of the same color as the background of the license plate. The background of the license plate is the pixels other than the pixels of the characters within the rectangle detected by the coordinates. The area where the license plate is expected to be located may also be recognized based on the number of pixels in the detected rectangle of the four characters.

[0048] The shape recognition unit 122 recognizes the license plate in the above manner and identifies the area on the image where the license plate is drawn. As a result, the recognition device 100 recognizes the shape of the license plate from the identified area and outputs the recognition result.

[0049] While the example of a license plate has been described, the target may also be road signs or road markings indicating speed limits, etc. In the case of the sign shown in FIG. 8, each character in a string of specific numbers, such as 30, is detected. Each character is then evaluated, and it can be determined to be a road sign if the color, font, and pixel value of the character on the concentric circle change from white to red, for example.

[0050] Next, the operation of the recognition device 100 will be described.

[0051] 10 is a flowchart showing the flow of the recognition process by the recognition device 100. The CPU 11 reads out the recognition program from the ROM 12 or the storage 14, loads it into the RAM 13, and executes it, thereby performing the recognition process.

[0052] In step S100, the CPU 11 functions as the acquisition unit 110 to acquire time-series images from the video captured by the on-board camera of the observation vehicle.

[0053] In step S102, the CPU 11 functions as the detection unit 120 to detect each character of the character string from the image using a detection rectangle.

[0054] In step S104, the CPU 11 functions as the shape recognition unit 122 to evaluate the positional relationship of the detected rectangles corresponding to the characters in the character string.

[0055] In step S106, the CPU 11 functions as the shape recognition unit 122 to evaluate the relationship between the pixels of the detected rectangle.

[0056] In step S108, the CPU 11 functions as the shape recognition unit 122 to identify the area on the image where the license plate is drawn, based on the evaluation results of the positional relationship and the evaluation results of the pixel relationship.

[0057] In step S110, the CPU 11, functioning as the shape recognition unit 122, recognizes the shape of the license plate from the identified area and outputs the recognition result.

[0058] As described above, according to the recognition device 100 of this embodiment, even if it is difficult to recognize a specific object, it is possible to evaluate the detected characters and recognize the object.

[0059] Although the example of recognizing characters including numerical values ​​using object detection technology has been described, other methods such as pattern matching may also be used. For example, multiple patterns are prepared and locations in the image that are similar to the patterns are searched for.

[0060] Among the characters used on license plates, dots are difficult to recognize and are used in Japan's unique license plate numbers. For example, instead of the rule of leading zeros like "00-08," there is a rule that leading zeros are represented by dots, such as "··-·8," "··-28," and "·1-28." Therefore, as a variation, object detection technology can be applied only to numbers, and dots can be searched for individually. If the detected number is a single character, the system searches for three pixel areas with patterns that could be interpreted as dots among the corresponding pixels on the "left," taking into account the hyphen area. The surrounding area that is a candidate for a dot is then binarized, and the dot is detected by determining whether the center is a black pixel and the rest are white pixels. Furthermore, if the detected number digit is two characters, the search range is two locations, and if it is three characters, the search range is one location.

[0061] Although an example using an in-vehicle camera has been described, any camera that captures the environment in which a vehicle is traveling may be used. It may be a fixed surveillance camera installed above an intersection or in a parking lot, or a camera for identifying speeding vehicles. It may also be a surveillance camera installed on a sidewalk or in a storefront. In such cases, since license plates and road signs may be captured at an angle rather than directly in front of them, the aspect ratio of the detected letters and numbers may be taken into account, and any stretched or curved parts may be corrected for shape recognition. Furthermore, when evaluating the relationship between detected rectangles, the aspect ratio of each target character may also be taken into account.

[0062] In the above embodiment, the recognition process executed by the CPU after reading the software (program) may be executed by various processors other than the CPU. Examples of such processors include programmable logic devices (PLDs) such as field-programmable gate arrays (FPGAs), whose circuit configuration can be changed after fabrication, and application-specific integrated circuits (ASICs), which are dedicated electrical circuits that are processors with circuit configurations specifically designed to execute specific processes. The recognition process may be executed by one of these processors, or by a combination of two or more processors of the same or different types (e.g., multiple FPGAs, or a combination of a CPU and an FPGA). The hardware structure of these processors is, more specifically, an electrical circuit that combines circuit elements such as semiconductor devices.

[0063] In the above embodiment, the recognition program is pre-stored (installed) in the storage 14, but the present invention is not limited to this. The program may be provided in a form stored in a non-transitory storage medium such as a CD-ROM (Compact Disk Read Only Memory), a DVD-ROM (Digital Versatile Disk Read Only Memory), or a USB (Universal Serial Bus) memory. The program may also be downloaded from an external device via a network.

[0064] The following additional notes are provided regarding the above-described embodiments.

[0065] (Additional note 1) Memory and at least one processor coupled to said memory; Including, The processor: Acquire a time series of images acquired in an environment in which a vehicle is traveling; Detecting each character of a predetermined string from the image; evaluating the relationship between each of the detected characters of the character string and recognizing the shape of an object containing the character string; A recognition device configured as follows.

[0066] (Additional note 2) A non-transitory storage medium storing a program executable by a computer to perform a recognition process, Acquire a time series of images acquired in an environment in which a vehicle is traveling; Detecting each character of a predetermined string from the image; evaluating the relationship between each of the detected characters of the character string and recognizing the shape of an object containing the character string; Non-transitory storage medium. [Explanation of symbols]

[0067] 100 recognition device 110 Acquisition Department 112 Recognition part 114 Storage section 120 Detector 122 Shape recognition section

Claims

1. an acquisition unit that acquires time-series images acquired in an environment in which a vehicle is traveling; a detection unit that detects each character of a predetermined character string from the image; a shape recognition unit that evaluates the relationship between each of the detected characters of the character string and recognizes the shape of an object that includes the character string; Including, As the image, an image captured by an in-vehicle camera is used, the detection unit detects a detection rectangle representing a range of the character defined by the coordinates of each of the characters; the shape recognition unit determines whether the fonts of each character of the character string are the same based on a specific font type and aspect ratio, evaluates the positional relationship between fonts determined to be the same, and recognizes the shape of the object by specifying an area on the image in which the object is drawn, using at least one of a license plate and a sign as the object; In recognizing the license plate, the positional relationship is evaluated, and the pixel relationship between character pixels and non-character pixels in the detection rectangle is evaluated, and if conditions are met, the plate is recognized as the license plate; In recognizing the sign, the recognition device detects a specific string of numbers, evaluates each character, and determines that it is a road sign if the color, font, and pixel values ​​on the concentric circles of the characters change.

2. The recognition device according to claim 1 , wherein the shape recognition unit evaluates a positional relationship between the characters.

3. 3. The recognition device according to claim 1, wherein the detection unit detects each character in the character string using a model that has been trained in advance to detect characters in a specific font.

4. Acquire a time series of images acquired in an environment in which a vehicle is traveling; Detecting each character of a predetermined string from the image; Evaluating the relationship between each of the detected characters of the character string and recognizing the shape of an object containing the character string; As the image, an image captured by an in-vehicle camera is used, The detecting step includes detecting a detection rectangle representing a range of the character defined by the coordinates of each of the characters; In the shape recognition process, it is determined whether the fonts of each character of the character string are the same based on a specific font type and aspect ratio, and the positional relationship between fonts determined to be the same is evaluated. At the same time, the shape of the object is recognized by specifying an area on the image in which at least one of a license plate and a sign is drawn as the object, and In recognizing the license plate, the positional relationship is evaluated, and the pixel relationship between character pixels and non-character pixels in the detection rectangle is evaluated, and if conditions are met, the plate is recognized as the license plate; The sign recognition involves detecting a specific string of numbers, evaluating each character, and determining that the character is a road sign if the character color, font, and pixel values ​​on the concentric circles change. A recognition method that causes a computer to perform processing.

5. Acquire a time series of images acquired in an environment in which a vehicle is traveling; Detecting each character of a predetermined string from the image; Evaluating the relationship between each of the detected characters of the character string and recognizing the shape of an object containing the character string; As the image, an image captured by an in-vehicle camera is used, The detecting step includes detecting a detection rectangle representing a range of the character defined by the coordinates of each of the characters; In the shape recognition process, it is determined whether the fonts of each character of the character string are the same based on a specific font type and aspect ratio, and the positional relationship between fonts determined to be the same is evaluated. At the same time, the shape of the object is recognized by specifying an area on the image in which at least one of a license plate and a sign is drawn as the object, and In recognizing the license plate, the positional relationship is evaluated, and the pixel relationship between character pixels and non-character pixels in the detection rectangle is evaluated, and if conditions are met, the plate is recognized as the license plate; The sign recognition involves detecting a specific string of numbers, evaluating each character, and determining that the character is a road sign if the character color, font, and pixel values ​​on the concentric circles change. A recognition program that causes a computer to perform processing.

Images