Pattern recognition device

Abstract

To provide a pattern identification device acquiring rail information to assist in operation of a vehicle.SOLUTION: A pattern identification device 5 includes a detection unit 524 for detecting a first region and a second region having a surface reflection luminance higher than that of the first region or a surface reflectance different from that of the first region in a landmark included in an input image of the front of a vehicle 1 from an imaging device 4 installed on the vehicle 1, a storage unit 53 for storing landmark pattern information identified from shapes and positions of the first region and the second region and rail information in an advancing direction of the vehicle 1 linked with the landmark pattern information, and a specification unit 525 for acquiring rail information corresponding to the landmark pattern information, which corresponds to the first region and the second region detected by the detection unit 524, from the storage unit 53.SELECTED DRAWING: Figure 9

Description

Technical Field

[0004] ,

[0006] , , , , , ,

[0005] , , ,

[0003] , , , , , ,

[0001] Embodiments of the present invention relate to a pattern recognition device.

Background Art

[0002] A method is known in which a landmark building is detected from a camera attached in front of a vehicle (particularly, a train), and the current vehicle position information is obtained by comparing it with the position information of the landmark stored in a database in advance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] For example, if it is possible to obtain information on the shape and gradient of the rail in front of the vehicle from the landmark, these pieces of information can be used to support the automatic driving of the vehicle and the operation of the driver.

[0005] Embodiments of the present invention have been made in view of the above circumstances, and an object thereof is to provide a pattern recognition device that acquires rail information for supporting the driving of a vehicle.

Means for Solving the Problems

[0006] A pattern recognition device according to one embodiment includes: a detection unit that detects a first region and a second region having a higher surface reflectivity or a different surface reflectivity than the first region in an input image of the front of the vehicle from an imaging device positioned on the vehicle; a storage unit that stores landmark pattern information identified from the shape and position of the first and second regions, and rail information in the direction of vehicle travel associated with the landmark pattern information; and a specification unit that acquires the rail information corresponding to the landmark pattern information corresponding to the first and second regions detected by the detection unit from the storage unit. The rail information includes rail shape information and gradient information, the detection unit detects the first and second regions of the first and second landmarks, the storage unit stores the rail shape information associated with first landmark pattern information identified from the shape and position of the first and second regions of the first landmark, and the gradient information associated with second landmark pattern information identified from the shape and position of the first and second regions of the second landmark, and the identification unit comprises a rail identification unit that acquires the rail shape information corresponding to the first landmark pattern information from the storage unit, and a gradient identification unit that acquires the gradient information corresponding to the second landmark pattern information. [Brief explanation of the drawing]

[0007] [Figure 1] Figure 1 is a schematic diagram showing an example of the overall configuration of a pattern identification system including a pattern identification device according to one embodiment. [Figure 2] Figure 2 is a block diagram showing an example of the configuration of a central device according to one embodiment. [Figure 3] Figure 3 shows an example of a landmark pattern. [Figure 4] Figure 4 shows an example of a landmark pattern. [Figure 5] Figure 5 shows an example of a landmark pattern. [Figure 6] Figure 6 shows an example of a landmark pattern. [Figure 7] Figure 7 shows an example of a landmark pattern. [Figure 8] Figure 8 shows an example of a landmark pattern. [Figure 9] Figure 9 is a block diagram showing an example configuration of a vehicle equipped with a pattern identification device according to one embodiment. [Figure 10] Figure 10 is a flowchart illustrating an example of the pattern identification processing operation of a pattern identification device according to one embodiment. [Figure 11]Figure 11 is a flowchart illustrating another example of the pattern identification processing operation of a pattern identification device according to one embodiment. [Modes for carrying out the invention]

[0008] The pattern identification device according to the embodiment will be described in detail below with reference to the drawings. Note that the scale of each part in the drawings used in the description of the embodiment below has been changed as appropriate. Also, in the drawings used in the description of the embodiment below, some components may be omitted for illustrative purposes.

[0009] Figure 1 is a schematic diagram showing an example of the overall configuration of a pattern identification system including a pattern identification device according to one embodiment.

[0010] The pattern recognition system shown in Figure 1 comprises a vehicle 1 and a central device 2. Vehicle 1 is, for example, a vehicle included in a train that runs on rails according to a predetermined schedule. Note that if a train has multiple vehicles, not all vehicles need to have the functions described below.

[0011] Vehicle 1 is connected to the central device 2 and satellite 3 via networks. Vehicle 1 is, for example, the vehicle located at the front of a train in the direction of travel, and is equipped with an antenna (not shown) that receives radio waves from satellite 3. For example, the antenna is installed on the roof approximately 3m behind the frontmost position of the vehicle that is designed to be at the very front of the train. The position information of vehicle 1 is based on the position where the antenna is installed. In addition, the position information of the imaging device 4 of vehicle 1 can be corrected based on the positional relationship between the imaging device 4 and the antenna. The central unit 2 is configured to communicate with the vehicles 1 via a network and, for example, manages the operating status of multiple vehicles 1.

[0012] Figure 2 is a block diagram showing an example of the configuration of the central device 2 according to one embodiment. The central device 2 of this embodiment includes a receiving unit 21, a control unit 22, an input unit 23, an output unit 24, a storage unit 25, a transmitting unit 26, and a bus communication line BL1. The bus communication line BL1 is connected to each of the components included in the central device 2. The control unit 22 can transmit and receive data to and from other components included in the central device 2 via the bus communication line BL1.

[0013] The receiving unit 21 receives landmark information and information regarding the error corrected by the vehicle 1 (hereinafter referred to as error information) from the vehicle 1. The error information includes, for example, that there is an error in the current position information of the vehicle 1 (hereinafter referred to as the first position information) using GNSS position information, and that the error has been corrected using the position information of the vehicle 1 (hereinafter referred to as the second position information) obtained using landmarks included in the input image from the imaging device 4 described later, the landmark information included in the corrected input image, and the correction area estimated from the first position information, at least one of them. The information included in the error information is not limited to the above. Details of the landmark information and the second position information will be described later.

[0014] The control unit 22 includes at least one processor such as a CPU (Central Process Unit), MPU (micro processing unit), GPU (Graphics Processing Unit), FPGA (field-programmable gate array). The control unit 22 can realize various functions of the central device 2 based on programs such as system software, application software, or firmware stored in the auxiliary storage unit 252.

[0015] The control unit 22 compares the landmark information fed back from the vehicle 1 with the landmark information included in the data of the map database stored in the storage unit 25 by referring to the position information of the vehicle 1. When the landmarks existing in the landmark candidate area included in the fed-back landmark information do not match the landmarks existing in the landmark candidate area included in the landmark information of the map database, the control unit 22 modifies the information as necessary and outputs it to the output unit 24.

[0016] The input unit 23 may include, for example, a user interface such as a mouse or a keyboard, and a microphone, a touch panel, a camera, and various sensors. The input unit 23 transmits the information acquired by the user's operation to the control unit 22 via the bus communication line BL1.

[0017] The output unit 24 may include, for example, display means such as a monitor, and audio output means such as a speaker. Note that the output unit 24 may be configured to be connected to the outside of the computer. The output unit 24 displays the information output by the control unit 22 on a monitor or the like as alarm information or outputs it using audio output means such as a speaker.

[0018] The storage unit 25 includes, for example, a main storage unit 251 and an auxiliary storage unit 252. The main storage unit 251 may include, for example, a ROM (read-only memory) and a RAM (random-access memory). The ROM is a non-volatile memory used exclusively for reading data, and can store data and various setting values used by the control unit 22 to perform various processes. In addition, the RAM can be used as a so-called work area for temporarily storing data when the control unit 22 performs various processes. The main storage unit 251 of the present embodiment is, for example, a RAM and is used as a memory. The main storage unit 251 can temporarily store the data of the map database, the position information priority setting, the error information, and the like.

[0019] The auxiliary storage unit 252 is a non-temporary computer-readable storage medium for the computer centered around the control unit 22. The auxiliary storage unit 252 is, for example, an EEPROM (electric erasable programmable read-only memory), an HDD (hard disk drive), or an SSD (solid state drive).

[0020] The auxiliary storage unit 252 can store data used by the control unit 22 in performing various processes, data generated by the processing in the control unit 22, or various setting values. For example, the auxiliary storage unit 252 is a memory for storing various information, and can store timetable (schedule) information, map database data, location information priority settings, and error information. A specific explanation of location information priority settings will be given later.

[0021] Timetable information includes, for example, information about the stations where the train stops, the time of arrival at each station, and the time of departure from each station, as well as information about the train's location and time along the route of train 1.

[0022] The map database contains multiple sets of data associated with and stored to correspond to the location information of vehicle 1. The map database contains multiple sets of data such as location information (latitude, longitude, altitude), land topography (gradient), rail shape, location features, landmark information (number, size, coordinates, type, landmark pattern information), left rail node information (number, coordinates), and right rail node information (number, coordinates). The landmark pattern information refers to information about patterns identified from the shape and position of multiple regions (hereinafter referred to as the first region and the second region) with different brightness (surface reflectivity) and surface reflectivity that exist within the landmark region. In this embodiment, the second region is assumed to have higher brightness (surface reflectivity) or a different surface reflectivity than the first region. As described later, the detection unit 524 distinguishes between the first and second regions of a landmark based on brightness (surface reflectance). Therefore, it is desirable that the first and second regions be formed such that there is a large difference in brightness (surface reflectance). For example, it is desirable that the first region be black so that its brightness (surface reflectance) is low, and the second region be a bright color so that its brightness (surface reflectance) is high.

[0023] Next, we will describe an example of the shape and location of the first and second regions included in the landmark pattern information. Figures 3 to 8 show examples of multiple landmark patterns. In the following description of landmark M, the vertical direction is referred to as the vertical direction, the horizontal direction as the horizontal direction, and the right and left sides refer to the right and left sides when approaching landmark M from vehicle 1.

[0024] The multiple landmarks M shown in Figures 3 to 8 are mainly mounted on overhead line poles and freestanding poles located outside the rails of the vehicle 1. In the example in Figure 3, for example, landmark M is formed on a rectangular base of approximately 80 cm in height and 60 cm in width. The size and shape of landmark M are not limited to those described above, but are designed in advance considering installation on overhead line poles and freestanding poles, and contact with the vehicle 1.

[0025] In Figure 3, the landmark M, which represents a straight line in the shape of a rail, has a first rectangular region M1 that includes a long side extending vertically. The first region M1 is smaller than the area of ​​landmark M and is located at the center of landmark M. The area of ​​landmark M other than the first region M1 is the second region M2, and the second region M2 is arranged to surround the first region M1.

[0026] Landmark M, which indicates a right curve in the rail shape, has a first region M1 that includes a long side extending vertically. The first region M1 is smaller than the region of landmark M, and is positioned so that the right end of the first region M1 and the right end of landmark M overlap. The region of landmark M other than the first region M1 is the second region M2, which is roughly U-shaped and touches the short side and the long side on the left of the first region M1.

[0027] Landmark M, which indicates a left curve in the rail shape, has a first region M1 that includes a long side extending vertically. The first region M1 is smaller than the region of landmark M, and is positioned so that the left end of the first region M1 and the left end of landmark M overlap. The region of landmark M other than the first region M1 is the second region M2, which is roughly U-shaped and touches the short side and the long side on the right of the first region M1. In other words, the pattern of landmark M indicating a left curve is the same as the pattern of landmark M indicating a right curve, rotated 180 degrees.

[0028] Landmark M, which indicates an uphill slope, comprises a first region M1 that includes a long side extending vertically. The first region M1 is smaller than the region of landmark M, and is positioned so that the upper end of the first region M1 and the upper end of landmark M overlap. The region of landmark M other than the first region M1 is the second region M2, which is roughly U-shaped and touches the lower short side and long side of the first region M1.

[0029] Landmark M, which indicates a downward slope, comprises a first region M1 that includes a long side extending vertically. The first region M1 is smaller than the region of landmark M, and is positioned so that the lower end of the first region M1 and the lower end of landmark M overlap. The region of landmark M other than the first region M1 is the second region M2, which is roughly U-shaped and touches the upper short side and long side of the first region M1. In other words, the pattern of landmark M indicating a downhill slope is the same as the pattern of landmark M indicating an uphill slope, rotated 180 degrees.

[0030] Landmark M, which indicates a rail-shaped branch, includes a roughly V-shaped first region M1. The first region M1 is positioned at a distance from the end of landmark M. The area of ​​landmark M other than the first region M1 is the second region M2, which is positioned to surround the first region M1.

[0031] Landmark M, which indicates the confluence of rail shapes, includes a roughly V-shaped first region M1. The first region M1 is positioned with a gap between it and the ends of landmark M, so that the V shape is inverted. The area of ​​landmark M other than the first region M1 is the second region M2, which is positioned to surround the first region M1. In other words, the pattern of landmark M indicating a confluence is the same as the pattern of landmark M indicating a divergence, rotated 180 degrees.

[0032] Figure 4 shows an example of representing rail shape and gradient using two landmarks (the first landmark and the second landmark). Here, by combining multiple landmark patterns as shown in Figure 3, flat straight sections, straight uphill sections, and downhill right curves are represented.

[0033] For example, a combination of landmark M indicating a flat straight line is a combination of two landmark M indicating a straight line in Figure 3. A combination of landmark M indicating an uphill straight line is a combination of two landmark M indicating an uphill slope in Figure 3. A combination of landmark M indicating a downhill straight line is a combination of two landmark M indicating a downhill slope in Figure 3. The combination of landmark M indicating a flat right curve is a combination of two landmark M indicating a right curve in Figure 3. The combination of landmark M indicating a flat left curve is a combination of two landmark M indicating a left curve in Figure 3.

[0034] The combination of landmark M indicating an uphill right curve is the combination of the landmark M indicating an uphill slope and the landmark M indicating a right curve in Figure 3. The combination of landmark M indicating a downhill right curve is the combination of the landmark M indicating a downhill slope and the landmark M indicating a right curve in Figure 3. The combination of landmark M indicating an uphill left curve is the combination of the landmark M indicating an uphill slope and the landmark M indicating a left curve in Figure 3. The combination of landmark M indicating a downhill left curve is the combination of the landmark M indicating a downhill slope and the landmark M indicating a left curve in Figure 3.

[0035] Figures 5 through 7 show how a single landmark is used to represent the rail shape and gradient. By combining the basic configurations, it is possible to represent both the rail shape and gradient with a single landmark M. In the examples shown in Figures 5 to 7, for example, landmark M is formed on a square base that includes vertically extending sides and horizontally extending sides. The size and shape of landmark M are designed in advance, taking into consideration installation on overhead line poles or freestanding poles, and contact with vehicles 1.

[0036] In the example shown in Figure 5, the landmark M, which represents a flat straight line, includes a first region M1 that contains vertically extending and horizontally extending edges. The first region M1 is smaller than the region of landmark M and is located at the center of landmark M. The region of landmark M other than the first region M1 is the second region M2. The second region M2 is positioned to surround the perimeter of the first region M1.

[0037] Landmark M indicating an upward slope is obtained by extending the first region M1 to the upper end of landmark M in the pattern of the flat, straight landmark M described above. Landmark M indicating a downward slope is obtained by extending the first region M1 to the lower end of landmark M in the pattern of the flat, straight landmark M described above.

[0038] Landmark M, which indicates a right curve, is obtained by extending the first region M1 to the rightmost end of landmark M in the pattern of the flat, straight landmark M described above. The landmark M indicating the left curve is obtained by extending the first region M1 to the left end of the second region M2 in the pattern of the flat straight landmark M described above.

[0039] For example, in the pattern of the flat, straight landmark M, a landmark M indicating that the rail shape is a left curve and the gradient is uphill, a square pattern similar to that of the first region M1 is arranged so as to be tangent to the left and top ends of the landmark M. Landmark M, which indicates that the rail shape is a right curve and the gradient is uphill, is positioned in the pattern of the flat straight landmark M described above so that a square pattern similar to that of the first region M1 is tangent to the right and top ends of landmark M.

[0040] Landmark M, which indicates that the rail shape is a left curve and the gradient is downward, is positioned in the pattern of the flat straight landmark M described above so that a square pattern similar to that of the first region M1 is tangent to the left and bottom ends of landmark M. Landmark M, which indicates that the rail shape is a right curve and the gradient is downward, is positioned in the pattern of the flat straight landmark M described above so that a square pattern similar to that of the first region M1 is tangent to the right and bottom ends of landmark M.

[0041] Figures 6 and 7 show variations in the pattern of landmark M shown in Figure 5. In Figures 6 and 7, the pattern of landmark M, which indicates the curve and the vertical slope, differs from the pattern shown in Figure 5. For example, a pattern of landmark M indicating a left curve and an uphill slope requires that the first region M1 extends between the center and the upper left corner of the landmark M region. A pattern of landmark M indicating a left curve and a downhill slope requires that the first region M1 extends between the center and the lower left corner of the landmark M region. Similarly, a pattern of landmark M indicating a right curve and an uphill slope requires that the first region M1 extends between the center and the upper right corner of the landmark M region. A pattern of landmark M indicating a right curve and a downhill slope requires that the first region M1 extends between the center and the lower right corner of the landmark M region.

[0042] In Figure 8, a single landmark M is used to represent the directions of confluence and divergence. Landmark M, which indicates the direction of merging and diverging, is formed on the base of a square, for example, with one diagonal extending vertically and the other diagonal extending horizontally. The size and shape of landmark M are designed in advance, taking into consideration installation on overhead line poles or freestanding poles, and contact with vehicles 1.

[0043] A landmark M, which represents a straight line without diversion, includes multiple first regions M1 arranged along a diagonal extending vertically from the landmark M. Each of the first regions M1 is a square pattern with a diagonal extending vertically, and multiple first regions M1 are arranged continuously between the upper corner and the lower corner of the landmark M. The area of ​​the landmark M other than the first regions M1 is the second region M2. Landmark M, which indicates a branching off to the left and a merger from the left, is a pattern of the straight line shown by landmark M, with a square first region M1 pattern further arranged along the rightmost corner of landmark M.

[0044] Landmark M, which indicates a branch to the right and a merge from the right, is a pattern of the straight line shown by landmark M, with a square first region M1 pattern further arranged along the leftmost corner of landmark M. The shapes and positions of the first and second regions can be freely set as long as the landmark pattern has a certain regularity, not limited to the example above.

[0045] In the map database, multiple data points are associated with identifiers (e.g., line numbers) that are assigned sequentially, for example, to the location information of a vehicle. However, the data contained in the map database is not limited to the above. The data in the map database is assumed to have been acquired or updated in advance by a dedicated vehicle.

[0046] The coordinates in the left and right rail node information included in the map database are the position coordinates of the rails in the image captured by the imaging device 4 when vehicle 1 is traveling at the location information (latitude, longitude, altitude). The node coordinates are calculated by setting the upper left corner of the input image from the imaging device 4 (upper left corner when facing the direction of travel of vehicle 1) as the reference 0 (origin), with the horizontal direction to the right of the reference being a positive value for the x-axis and the vertical direction downward from the reference being a positive value for the y-axis. The reference point and axis settings are assumed to be predetermined by the map database creator, etc. The same applies to the coordinates in the landmark information held in the map database.

[0047] The transmission unit 26 transmits the map database data to the vehicle 1 (the pattern identification device 5 and support control device 6 described later). The map database data in the central device 2 may consist of two map databases: master data and vehicle instruction data. The master data of the map database includes detailed map information for the entire driving route, and the vehicle instruction data of the map database includes information from at least a portion of the master data, including data for each vehicle's driving route in accordance with the timetable. The transmission unit 26 transmits at least the vehicle instruction data to the pattern identification device 5 and support control device 6 of the vehicle 1 in response to a request from the vehicle 1, or periodically.

[0048] Figure 9 is a block diagram showing an example configuration of a vehicle equipped with a pattern identification device according to one embodiment.

[0049] The vehicle 1 of this embodiment includes an imaging device 4, a pattern recognition device 5, a support control device 6, and a bus communication line BL2. The bus communication line BL2 is connected to each of the multiple components included in the vehicle 1. The control unit 52 included in the pattern identification device 5 and the correction control unit 62 included in the support control device 6 can communicate data with the other components included in the vehicle 1 via the bus communication line BL2.

[0050] The imaging device 4 is, for example, a stereo camera that photographs the area in front of the vehicle 1. The imaging device 4 transmits the captured image as an input image to the pattern recognition device 5. The imaging device 4 may be configured to transmit the input image 30 times per second when the frame rate is 30 fps. The frequency at which the imaging device 4 sends the input image to the pattern recognition device 5 is not limited to the above and can be appropriately changed depending on the frame rate of the imaging device 4 and the frequency of radio wave reception from satellite 3. The imaging device 4 may also include, for example, a LiDAR or distance sensor to detect the distance to an object in front of the vehicle 1 and transmit the detected value to the pattern recognition device 5. However, it is not possible to directly detect and identify the pattern of a landmark from the distance information acquired by the LiDAR or TOF distance sensor. Therefore, it is conceivable to input and process the difference in reflectivity of the object surface acquired by these sensors (digitized, converted into an image, or 2D data) into the pattern recognition device 5 so that the pattern of the landmark can be detected and identified.

[0051] The support control device 6 comprises a receiving unit 61, a correction control unit 62, a storage unit 63, and a transmitting unit 64.

[0052] The receiving unit 61 receives first position information from the antenna that receives radio waves from satellite 3. The receiving unit 61 has the function of receiving multiple data from the map database. The receiving unit 61 also receives second position information and landmark information from the pattern recognition device 5.

[0053] The correction control unit 62 includes at least one processor such as a CPU (Central Process Unit), MPU (microprocessing unit), GPU (Graphics Processing Unit), or FPGA (field-programmable gate array). The correction control unit 62 can implement various functions of the support control device 6 based on programs such as system software, application software, or firmware stored in the auxiliary storage unit 632.

[0054] The correction control unit 62 compares the first position information with the second position information and performs processing according to the comparison result. More specifically, the correction control unit 62 detects the error between the first position information and the second position information, and if the error exceeds a predetermined threshold, it can refer to the position information priority setting and correct the error by using either the first or second position information as the current position information of vehicle 1.

[0055] The memory unit 63 includes, for example, a main memory unit 531 and an auxiliary memory unit 532. The main memory unit 631 may include, for example, ROM (read-only memory) and RAM (random-access memory). ROM is a non-volatile memory used exclusively for reading data, and can store data and various setting values ​​used by the correction control unit 62 in performing various processes. RAM can be used as a so-called work area to temporarily store data when the correction control unit 62 performs various processes. In this embodiment, the main memory unit 631 is, for example, RAM and is used as memory.

[0056] The main memory unit 631 can temporarily store data from the map database, first location information, second location information, landmark information, location information priority setting, and information regarding errors corrected by the correction control unit 62 (error information). The auxiliary storage unit 632 is a non-temporary computer-readable storage medium of the computer centered around the correction control unit 62. The auxiliary storage unit 632 is, for example, an EEPROM (electric erasable programmable read-only memory), an HDD (hard disk drive), or an SSD (solid state drive).

[0057] The auxiliary storage unit 632 can store data used by the correction control unit 62 in performing various processes, data generated by processing in the correction control unit 62, or various setting values. For example, the auxiliary storage unit 632 is a memory for storing various information, and can store data from the map database, first location information, second location information, landmark information, location information priority settings, and information related to errors corrected by the correction control unit 62 (error information). The transmitting unit 64 transmits landmark information and error information to the central device 2. The transmitting unit 64 also transmits the error information to the pattern identification device 5.

[0058] The pattern identification device 5 of this embodiment comprises a receiving unit 51, a control unit 52, a storage unit 53, and a transmitting unit 54. The pattern recognition device 5 may include a processor that executes a program to realize various functions described later, and a memory that stores the program. The processor is typically a CPU (Central Processing Unit) and / or a GPU (Graphics Processing Unit), but may also be a microcontroller, FPGA (Field Programmable Gate Array), or DSP (Digital Signal Processor). The memory stores the program executed by the processor to realize the operation of the pattern recognition device 5, and also temporarily stores data used by the processor. The program may be recorded on a recording medium readable by the pattern recognition device 5. In that case, the processor can realize various functions by executing the program read from the recording medium.

[0059] The receiving unit 51 receives first position information from an antenna that receives radio waves from satellite 3. The receiving unit 51 is equipped with the function to receive multiple timetable information and map database data from the central device 2. For example, the receiving unit 51 can receive a day's worth of timetable and map database data from the central device 2 all at once before starting the day's journey.

[0060] The receiving unit 51 can receive map database data according to the operating status. The map database data received by the receiving unit 51 may be configured such that, for example, when vehicle 1 moves from station A to station B, it receives data from station A to station B from the central device 2, and then when it moves from station B to station C, it receives data from station B to station C from the central device 2. Alternatively, for example, the map database data received by the receiving unit 51 from the central device 2 may only receive data associated with location information of a few meters before and after the current location of vehicle 1, and update this data in real time.

[0061] The receiving unit 51 further receives input images from the imaging device 4. The imaging device 4 and the receiving unit 51 may be connected via a wired connection or via a wireless connection. The receiving unit 51 can communicate with the imaging device 4 based on communication standards such as the Internet, Ethernet (registered trademark), wireless LAN (Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.).

[0062] The control unit 52 includes at least one processor, such as a CPU (Central Process Unit), MPU (microprocessing unit), GPU (Graphics Processing Unit), or FPGA (field-programmable gate array). The control unit 52 can implement various functions of the pattern identification device 5 based on programs such as system software, application software, or firmware stored in the auxiliary storage unit 532.

[0063] The control unit 52 includes a identification unit 525, a vehicle position acquisition unit 523, and a detection unit 524. The identification unit 525 includes a gradient identification unit 521 and a rail identification unit 522.

[0064] The detection unit 524 detects objects in the input image and depth image from the imaging device 4 that are real estate and have distinctive features such as shape, brightness (surface reflectivity), color, and pattern (landmarks). Landmark information is pre-included in the map database data and stored in association with rail information. The rail information includes at least one of rail shape information and gradient information.

[0065] The detection unit 524 refers to the landmark information associated with the first position information, and after recognizing the presence of a landmark, displays a candidate landmark area. The data contained in the landmark information includes, for example, the number of landmarks, their size (length and width), coordinates (x, y), landmark type (such as those attached to overhead line poles or freestanding poles, or buildings), and landmark pattern data.

[0066] The detection unit 524 detects landmarks within the landmark candidate region and divides the region according to the brightness (surface reflectivity) of the landmark's shape and pattern. For example, the detection unit 524 detects a landmark included in the input image of the front of the vehicle from an imaging device positioned on the vehicle, and identifies a first region and a second region that has higher brightness (surface reflectivity) or a different surface reflectivity than the first region.

[0067] The identification unit 525 identifies landmark pattern information corresponding to the first and second regions detected by the detection unit 524, and acquires rail information corresponding to the identified landmark pattern information from the storage unit 53. The rail information acquired by the identification unit 525 includes at least one of rail shape information and gradient information. The identification unit 525 can acquire at least one of the rail shape and gradient information based on the landmark from the storage unit 53. The identification unit 525 transmits the acquired rail information to the vehicle position acquisition unit 523. The identification unit 525 may also transmit the acquired rail information to the support control device 6. The vehicle 1 may use the rail information for speed control, etc.

[0068] The rail identification unit 522 identifies landmark pattern information corresponding to the first and second regions for the landmarks detected by the detection unit 524, and obtains rail shape information corresponding to the identified landmark pattern information from the storage unit 53. The gradient identification unit 521 identifies landmark pattern information corresponding to the first and second regions for the landmarks detected by the detection unit 524, and obtains gradient information corresponding to the identified landmark pattern information from the storage unit 53.

[0069] The vehicle position acquisition unit 523 can acquire position information using the data identified by the rail identification unit 522. The position information acquired by the vehicle position acquisition unit 523 becomes the second position information. The vehicle position acquisition unit 523 acquires the current vehicle position information (second position information) from the storage unit 53 based on the rail information acquired by the identification unit 525. Note that the rail information and position information are linked in the map database data. For example, the vehicle position acquisition unit 523 may acquire the current position information (second position information) from the storage unit 53 based on the rail shape information acquired by the rail identification unit 522, or it may acquire the current position information (second position information) from the storage unit 53 based on the gradient information acquired by the gradient identification unit 521, or it may acquire the current position information (second position information) from the storage unit 53 based on the first position information and the rail information.

[0070] The vehicle position acquisition unit 523 compares the first position information and the second position information. If there is an error between the first and second position information, it selects the position information according to the position information priority setting, which is predetermined based on the rail shape and other factors. Alternatively, the vehicle position acquisition unit 523 may not perform the process of selecting the position information of vehicle 1; instead, the support control device 6 may correct the error, and the pattern identification device 5 may execute the process based on the corrected result. The location information priority setting based on the rail shape described above is just one example and is not limited to this. The location information priority setting can be determined using the reliability of the first and second location information as one of the indicators.

[0071] In this embodiment, the positioning methods using GNSS include standalone positioning, relative positioning, and autonomous position detection. Standalone positioning involves receiving information such as the satellite's position and time transmitted from a GNSS satellite with a single antenna, measuring the time it takes for the radio waves to reach the receiver after they are transmitted from the satellite, and converting this into distance. The position of vehicle 1 is determined by simultaneously knowing the distances from four or more satellites to the observation point, using a GNSS satellite whose position is known as a moving reference point.

[0072] Relative positioning involves using two or more receivers to simultaneously observe four or more identical GNSS satellites. Using the position of the GNSS satellite as a reference, the relative positional relationship between two points is calculated by measuring the time difference between when the radio signals from the GNSS satellite reach each receiver. Autonomous position detection calculates position information using, for example, measurements from three axes of inertial sensors, wheel rotation speed, and current vehicle speed, when estimating the vehicle's position in areas where radio waves from GNSS satellites cannot reach.

[0073] In terms of location reliability, standalone positioning using L6 augmentation signals offers the highest reliability, followed by relative positioning, then standalone positioning without augmentation signals, and finally autonomous position detection. Relative positioning has a location error of approximately 0.5m or less, standalone positioning has an error of approximately 1-2m or less, and autonomous position detection is even better. Therefore, location information priority settings are configured according to the positioning method. These location information priority settings are predetermined by the administrator or user of the location detection system.

[0074] Let's explain a specific example of the above location information priority setting. If vehicle 1 is traveling at 108 km / h (30 m / s) and the frame rate of imaging device 4 is 30 fps, the distance traveled in one frame will be approximately 1 m. If vehicle 1 is traveling at 36 km / h (10 m / s) and the frame rate of imaging device 4 is 30 fps, the distance traveled in one frame will be approximately 0.3 m. As such, the distance traveled in one frame changes depending on the speed of vehicle 1, and therefore the amount of deviation in vehicle 1's location information also changes. For this reason, the location information priority setting will be set so that, for example, the first location information is prioritized if relative speed is used, and the second location information is prioritized if standalone positioning or autonomous position detection is used.

[0075] Furthermore, in areas where radio waves cannot reach, such as tunnels, the first position information is determined by autonomous position detection, but it is set according to the shape of the rails, as described above. The second position information is more reliable than the first position information and is set to have the highest priority regardless of the shape of the rails.

[0076] The memory unit 53 includes, for example, a main memory unit 531 and an auxiliary memory unit 532. The main memory unit 531 may include, for example, ROM (read-only memory) and RAM (random-access memory). ROM is a non-volatile memory used exclusively for reading data, and can store data and various setting values ​​used by the control unit 52 in performing various processes. RAM can be used as a so-called work area to temporarily store data when the control unit 52 performs various processes. In this embodiment, the main memory unit 531 is, for example, RAM and is used as memory.

[0077] The main memory unit 531 can temporarily store data from the map database, first location information, input images from the imaging device 4, second location information, landmark information, location information priority settings, landmark pattern information identified from the shape and position of the first and second regions included in the landmark information, rail information in the direction of vehicle travel associated with the landmark pattern information, and second location information associated with the rail information.

[0078] The auxiliary storage unit 532 is a non-temporary computer-readable storage medium for the computer centered around the control unit 52. The auxiliary storage unit 532 is, for example, an EEPROM (electric erasable programmable read-only memory), an HDD (hard disk drive), or an SSD (solid state drive). The auxiliary storage unit 532 can store data used by the control unit 52 in performing various processes, data generated by processing in the control unit 52, or various setting values. For example, the auxiliary storage unit 532 is a memory for storing various information and can store data from a map database, first position information, input images from the imaging device 4, landmark information, position information priority settings, landmark pattern information identified from the shape and position of the first and second regions included in the landmark information, rail information in the direction of vehicle travel associated with the landmark pattern information, and second position information associated with the rail information.

[0079] The transmitting unit 54 transmits the second location information and landmark information to the support control device 6. The transmitting unit 54 can communicate based on communication standards such as the Internet, Ethernet (registered trademark), wireless LAN (Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.).

[0080] Figure 10 is a flowchart illustrating an example of the pattern identification processing operation of a pattern identification device according to one embodiment. The following describes an example of a procedure for acquiring the first and second regions of a landmark using the pattern recognition device 5, and obtaining corresponding landmark pattern information and rail information. Note that the processing described below is just an example, and various processes that can achieve similar effects can be used as appropriate.

[0081] In Figure 10, the pattern identification device 5 receives, for example, vehicle instruction data (hereinafter referred to as "data") from the map database for a specific section from the central device 2 using the receiving unit 51. The pattern identification device 5 stores the data in the storage unit 53, and the control unit 52 retrieves the data from the storage unit 53 (step S1). Since the vehicle 1 in this embodiment is a vehicle that travels along a predetermined route according to the timetable, the time it takes to travel between predetermined stations is predetermined, and the data is acquired based on that time. The control unit 52 acquires input images from the imaging device 4 according to the number of frames (step S2). The input images are assumed to be acquired by the control unit 52 in real time.

[0082] The detection unit 524 determines whether a landmark has been detected (step S3). As a method for detecting a landmark, for example, it refers to landmark information associated with the first position information and displays a landmark candidate region. If there is no landmark in the landmark candidate region, the detection unit 524 determines NO in step S3 and returns to the state before processing S2. In other words, the detection unit 524 repeats processing S2 and S3 until a landmark is detected. On the other hand, if there is a landmark in the landmark candidate region, the detection unit 524 determines YES in step S3 and proceeds to the next process.

[0083] The detection unit 524 detects a first region of the landmark (step S4). Next, the detection unit 524 detects a second region of the landmark (step S5). The region detection in steps S4 and S5 may be performed at the same time, or the regions may be detected with a time difference.

[0084] The identification unit 525 acquires landmark pattern information corresponding to the first and second regions stored in the storage unit 53 (step S6). The specific unit 525 obtains rail information corresponding to the landmark pattern information from the storage unit 53 (step S7), and then terminates the process.

[0085] Figure 11 is a flowchart illustrating another example of the pattern identification processing operation of a pattern identification device according to one embodiment. In Figure 11, the pattern identification device 5 receives, for example, vehicle instruction data (hereinafter referred to as "data") from the map database for a specific section from the central device 2 using the receiving unit 51. The pattern identification device 5 stores the data in the storage unit 53, and the control unit 52 retrieves the data from the storage unit 53 (step S8).

[0086] The control unit 52 acquires input images from the imaging device 4 according to the number of frames (step S9). The input images are assumed to be acquired by the control unit 52 in real time. The detection unit 524 determines whether it has detected the first landmark (step S10). If there is no landmark in the landmark candidate region, the detection unit 524 determines NO in step S10 and returns to the state before processing S9. In other words, the detection unit 524 repeats processing S9 and S10 until it detects a landmark. On the other hand, if there is a first landmark in the landmark candidate region, the detection unit 524 determines YES in step S10 and proceeds to the next process.

[0087] The detection unit 524 detects a first region of the first landmark (step S11). Next, the detection unit 524 detects a second region of the first landmark (step S12). The region detection in steps S11 and S12 may be performed at the same time, or the regions may be detected with a time difference.

[0088] The rail identification unit 522 acquires landmark pattern information (first landmark pattern information) corresponding to the first and second regions stored in the memory unit 53 (step S13). The rail identification unit 522 obtains rail shape information corresponding to the landmark pattern information from the storage unit 53 (step S14), and then proceeds to the next process.

[0089] The detection unit 524 determines whether it has detected the second landmark (step S15). If the second landmark is in the landmark candidate region, the detection unit 524 determines YES in step S15 and proceeds to the next process. The detection unit 524 detects the first region of the second landmark (step S16). Next, the detection unit 524 detects the second region of the second landmark (step S17).

[0090] The gradient identification unit 521 acquires landmark pattern information (second landmark pattern information) corresponding to the first and second regions stored in the memory unit 53 (step S18). The gradient identification unit 521 obtains rail shape information corresponding to the landmark pattern information from the storage unit 53 (step S19), and then terminates the process.

[0091] According to the pattern identification device 5 of this embodiment, by acquiring rail shape and gradient information from landmark pattern information, the pattern identification device 5 can, for example, perform control such as reducing the vehicle's speed in advance if the shape of the rail ahead is a curve and it is on a downward slope. In other words, according to this embodiment, a pattern identification device can be provided that acquires rail information to support the operation of a vehicle.

[0092] While several embodiments of the present invention have been described, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These novel embodiments can be carried out in a variety of other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims of the invention and its equivalents.

[0093] In the above embodiment, the landmark pattern information and the rail information in the direction of vehicle travel associated with the landmark pattern information were included in the map database. However, the pattern identification device of the embodiment is not limited to this configuration, and the landmark pattern information and rail information may be pre-recorded in the storage unit 53 of the vehicle 1. Even in that case, the same effects as in the above embodiment can be obtained. The original claims of this application are included below. [C1] A detection unit that detects a first region and a second region having a higher surface reflectance or a different surface reflectance than the first region in a landmark included in the input image of the front of the vehicle from an imaging device positioned on the vehicle, A storage unit that stores landmark pattern information identified from the shape and position of the first and second regions, and rail information in the direction of vehicle travel associated with the landmark pattern information, A specific unit acquires rail information corresponding to the landmark pattern information corresponding to the first and second regions detected by the detection unit from the storage unit, A pattern identification device equipped with the following features. [C2] The aforementioned rail information includes rail shape information and gradient information. The detection unit detects the first region and the second region of the first landmark and the second landmark, The storage unit stores rail shape information associated with first landmark pattern information identified from the shape and position of the first and second regions of the first landmark, and gradient information associated with second landmark pattern information identified from the shape and position of the first and second regions of the second landmark. The pattern identification device according to C1, wherein the identification unit comprises a rail identification unit that acquires rail shape information corresponding to the first landmark pattern information from the storage unit, and a gradient identification unit that acquires gradient information corresponding to the second landmark pattern information. [C3] The aforementioned rail information includes rail shape information and gradient information. The pattern identification device according to C1, wherein the identifying unit acquires the rail shape information and gradient information corresponding to the landmark pattern information from the storage unit. [C4] The memory unit further stores position information associated with the rail information, A vehicle position acquisition unit acquires the current vehicle position information from the storage unit based on the rail information acquired by the specified unit. A pattern identification device according to C1, comprising: [Explanation of Symbols]

[0094] 1...Vehicle, 2...Central unit, 21...Receiver, 22...Control unit, 23...Input unit, 24...Output unit, 25...Storage unit, 251...Main storage unit, 252...Auxiliary storage unit, 26...Transmitter, 3...Satellite, 4...Imaging device, 5...Pattern recognition device, 51...Receiver, 52...Control unit, 521...Gradient identification unit, 522...Rail identification unit, 523...Vehicle position acquisition unit, 524...Detection unit, 525...Identification unit, 53...Storage unit, 531...Main storage unit, 532...Auxiliary storage unit, 54...Transmitter, 6...Support control device, 61...Receiver, 62...Correction control unit, 63...Storage unit, 631...Main storage unit, 632...Auxiliary storage unit, 64...Transmitter, BL1...Bus communication line, BL2...Bus communication line, M...Landmark, M1...First area, M2...Second area

Claims

1. A detection unit that detects a first region and a second region having a higher surface reflectance or a different surface reflectance than the first region in a landmark included in the input image of the front of the vehicle from an imaging device positioned on the vehicle, A storage unit that stores landmark pattern information identified from the shape and position of the first and second regions, and rail information in the direction of vehicle travel associated with the landmark pattern information, A specific unit acquires rail information corresponding to the landmark pattern information corresponding to the first and second regions detected by the detection unit from the storage unit, Equipped with, The aforementioned rail information includes rail shape information and gradient information. The detection unit detects the first region and the second region of the first landmark and the second landmark, The storage unit stores rail shape information associated with first landmark pattern information identified from the shape and position of the first and second regions of the first landmark, and gradient information associated with second landmark pattern information identified from the shape and position of the first and second regions of the second landmark. The identification unit includes a rail identification unit that acquires rail shape information corresponding to the first landmark pattern information from the storage unit, and a gradient identification unit that acquires gradient information corresponding to the second landmark pattern information. Pattern recognition device.

2. The aforementioned rail information includes rail shape information and gradient information. The pattern identification device according to claim 1, wherein the identifying unit acquires the rail shape information and gradient information corresponding to the landmark pattern information from the storage unit.

3. The memory unit further stores position information associated with the rail information, A vehicle position acquisition unit acquires the current vehicle position information from the storage unit based on the rail information acquired by the specified unit. A pattern identification device according to claim 1, comprising:

Images