Information processing apparatus and program

Abstract

An information processing apparatus and a program that efficiently recognize a line from an image are provided. According to an embodiment, the information processing apparatus includes an image interface, a communication interface, and a processor. The image interface acquires a photographed image including a line and a device. The communication interface acquires line information indicating geodetic coordinates of the line and device information indicating geodetic coordinates of the device. The processor determines a device region in which the device is photographed from the photographed image, generates a transformation matrix that establishes a correspondence between geodetic coordinates and coordinates in the photographed image based on coordinates of the device region in the photographed image and the geodetic coordinates of the device, transforms the geodetic coordinates of the line into coordinates in the photographed image based on the transformation matrix, and determines coordinates of the line in the photographed image.

Description

Technical Field

[0001] The embodiments of the present invention relate to information processing apparatus and programs. Background Technology

[0002] Among the vehicles traveling on the road, some use cameras to photograph the road ahead and detect obstacles. The vehicles identify their own path from the images captured by the cameras. Based on the identified path, the vehicles determine the areas they have traversed in the images. The vehicles then use images within those determined areas to detect obstacles.

[0003] For tasks such as obstacle detection, there is a desire for techniques that can effectively identify lines from images.

[0004] [Existing Technical Documents]

[0005] [Patent Literature]

[0006] [Patent Document 1] Japanese Patent Application Publication No. 2020-164013 Summary of the Invention

[0007] To solve the above-mentioned technical problems, an information processing device and program for effectively identifying lines from images are provided.

[0008] According to an embodiment, the information processing device includes an image interface, a communication interface, and a processor. The image interface acquires a photographic image including a line and equipment. The communication interface acquires line information represented by geodesic coordinates of the line and equipment information represented by geodesic coordinates of the equipment. The processor determines the equipment area where the equipment is captured from the photographic image, and based on the coordinates of the equipment area in the photographic image and the geodesic coordinates of the equipment, generates a transformation matrix that establishes a correspondence between the geodesic coordinates and the coordinates in the photographic image. Based on the transformation matrix, the geodesic coordinates of the line are transformed into the coordinates in the photographic image, thereby determining the coordinates of the line in the photographic image. Attached Figure Description

[0009] Figure 1 This is a schematic diagram of the vehicle system according to the first embodiment.

[0010] Figure 2 This is a block diagram illustrating the control system of the vehicle according to the first embodiment.

[0011] Figure 3 This is a diagram showing an example of device information in the first embodiment.

[0012] Figure 4 This is a diagram illustrating an example of the vehicle's operation according to the first embodiment.

[0013] Figure 5This is a diagram showing the route projected by the vehicle in the first embodiment.

[0014] Figure 6 This is a diagram representing the feature quantities of the photographic image of the first embodiment.

[0015] Figure 7 This is a flowchart illustrating an example of the vehicle's operation in the first embodiment.

[0016] Figure 8 This is a schematic diagram of the route, etc., taken by the vehicle in the second embodiment.

[0017] Figure 9 This is a top view of the route, etc., traversed by the vehicle in the second embodiment.

[0018] Figure 10 This is a diagram illustrating an example of device information in the second embodiment.

[0019] Figure 11 This is a flowchart illustrating an example of the vehicle's operation in the second embodiment. Detailed Implementation

[0020] The embodiments will now be described with reference to the accompanying drawings.

[0021] (First Implementation)

[0022] The vehicle system of this embodiment includes vehicles constituting a train traveling on a track. The vehicle takes photographs of the path ahead, including the track it is traveling on within the photographic area. The vehicle identifies the track from the photographed images. Based on the identified track's position, the vehicle sets its travel area within the photographed images. The vehicle then detects obstacles that may impede its travel based on the set travel area.

[0023] Figure 1 This is a schematic diagram of the vehicle system 100 according to the implementation method. (As shown) Figure 1 As shown, the vehicle system 100 consists of a vehicle 10 and a host device 20, etc. The vehicle 10 and the host device 20 are connected in a manner that enables them to communicate with each other.

[0024] Vehicle 10 constitutes a train that travels on line R. Vehicle 10 travels on line R through driver operation or automatic driving.

[0025] The vehicle 10 includes a shell 1, a camera 2, and an antenna 3.

[0026] The housing 1 forms the shape of the vehicle 10. For example, the housing 1 is configured to accommodate people or luggage inside.

[0027] A camera 2 is installed at the front of the housing 1. The camera 2 photographs the direction of travel of the vehicle 10. In addition, the camera 2 photographs the path R along which the vehicle 10 travels and the surrounding area of ​​the path R.

[0028] For example, camera 2 is a CCD (Charge Coupled Device) camera. Additionally, camera 2 may also include lights that illuminate the front of vehicle 10.

[0029] Antenna 3 is an antenna used to receive signals for positioning the vehicle 10. For example, antenna 3 is an antenna used to receive GNSS (Global Navigation Satellite System) signals.

[0030] The host device 20 provides various information to the vehicle 10. For example, the host device 20 may be a server managed by the railway company.

[0031] The host device 20 sends equipment information related to the equipment that the camera 2 of the vehicle 10 may photograph (such as utility poles, speed signs, or stop signs).

[0032] For example, the host device 20 receives a request for device information from the vehicle 10. The request includes the current coordinates of the vehicle 10 (e.g., latitude and longitude). Upon receiving the request, the host device 20 determines the device that the camera 2 may be photographing from the current coordinates included in the request. If the device is determined, the host device 20 sends device information related to the determined device to the vehicle 10 as a response to the request. The device information will be described in detail later.

[0033] In addition, the host device 20 sends route information to the vehicle 10, which represents the geodetic coordinates (coordinates measured by GNSS, etc.) of the route R that the vehicle 10 is scheduled to travel on.

[0034] For example, the host device 20 receives a request for equipment information from the vehicle 10. When the request is received, the host device 20 determines the route R that the vehicle 10 is traveling on. When the route R is determined, the host device 20 sends a response to the vehicle 10, including route information representing the geodetic coordinates of the determined route R.

[0035] For example, route information represents a series of geodetic coordinates (multiple geodetic coordinates) representing each point on route R.

[0036] In addition, the host device 20 will send boundary information, which represents the vehicle boundaries and building boundaries in line R, to vehicle 10.

[0037] Vehicle boundaries are the limits governing the size of the vehicle body's cross-section.

[0038] Building boundaries are the gaps in the road where no buildings are permitted.

[0039] For example, boundary information represents the vehicle boundaries and building boundaries at each geodetic coordinate of line R.

[0040] The host device 20 receives a request for boundary information from the vehicle 10. When the request is received, the host device 20 sends a response containing the boundary information to the vehicle 10.

[0041] Next, vehicle 10 will be described.

[0042] Figure 2 Example of the structure of vehicle 10 in the embodiment. Figure 2 This is a block diagram representing a structural example of vehicle 10. For example... Figure 2 As shown, vehicle 10 is equipped with camera 2, antenna 3 and information processing device 4, etc.

[0043] The information processing device 4 determines the coordinate system of the route R based on the photographic images from the camera 2. Additionally, the information processing device 4 detects obstacles that may impede the vehicle's movement.

[0044] The information processing device 4 includes a processor 11, ROM 12, RAM 13, NVM 14, communication unit 15, operation unit 16, display unit 17, camera interface 18, and antenna interface 19.

[0045] The processor 11, ROM 12, RAM 13, NVM 14, communication unit 15, operation unit 16, display unit 17, camera interface 18, and antenna interface 19 are interconnected via a data bus, etc. Furthermore, the camera interface 18 is connected to the camera 2. Additionally, the antenna interface 19 is connected to the antenna 3.

[0046] In addition, vehicle 10 and information processing device 4 Figure 2 In addition to the structure shown, other structures may be provided as needed, or specific structures may be excluded from vehicle 10.

[0047] The processor 11 has the function of controlling the overall movement of the vehicle 10. The processor 11 may also have internal cache and various interfaces. The processor 11 performs various processes by executing programs pre-stored in internal memory, ROM 12 or NVM 14.

[0048] Furthermore, some of the various functions implemented by the processor 11 in executing programs can also be implemented through hardware circuitry. In this case, the processor 11 controls the functions executed by the hardware circuitry.

[0049] ROM12 is a non-volatile memory that pre-stores control programs and control data. The control programs and control data stored in ROM12 are pre-loaded according to the specifications of vehicle 10.

[0050] RAM 13 is volatile memory. RAM 13 temporarily stores data processed by processor 11. RAM 13 saves various application programs according to commands from processor 11. In addition, RAM 13 can also save data required for executing application programs and the execution results of application programs.

[0051] NVM14 is a non-volatile memory capable of writing and rewriting data. NVM14 is composed of, for example, HDD (Hard Disk Drive), SSD (Solid State Drive), or flash memory. NVM14 stores control programs, applications, and various data depending on the application of the vehicle 10.

[0052] The communication unit 15 (communication interface) is the interface for sending and receiving data with the host device 20, etc. For example, the communication unit 15 is connected to the host device 20, etc. via a network. For example, the communication unit 15 is an interface that supports wired or wireless LAN (Local Area Network) connections.

[0053] The operation unit 16 receives various operation inputs from the operator (e.g., driver). The operation unit 16 sends a signal representing the input operation to the processor 11. The operation unit 16 may also be configured as a touch panel.

[0054] Display unit 17 displays image data from processor 11. For example, display unit 17 may be a liquid crystal monitor. If operation unit 16 is a touch panel, display unit 17 may also be integrally formed with operation unit 16.

[0055] Camera interface 18 (image interface) is the interface connected to camera 2. Camera interface 18 sends signals from processor 11 to camera 2. In addition, camera interface 18 receives signals (captured images, etc.) from camera 2 and sends them to processor 11.

[0056] Antenna interface 19 is an interface connected to antenna 3. Antenna interface 19 determines the current position (geodetic coordinates) of vehicle 10 based on signals from antenna 3, etc.

[0057] For example, antenna interface 19 determines the position based on the signal from antenna 3. When determining the position, antenna interface 19 fits the determined position onto a nearby road R based on map information including the position of road R. Antenna interface 19 obtains the fitted position as the current position of vehicle 10.

[0058] Antenna interface 19 sends the acquired current position to processor 11.

[0059] In addition, the functions of the antenna interface 19 can also be implemented by the processor 11.

[0060] Furthermore, when the vehicle 10 is driven by autonomous driving, the information processing device 4 may not have an operation unit 16 and a display unit 17.

[0061] Next, the functions implemented by vehicle 10 will be explained. The functions implemented by vehicle 10 are achieved by processor 11 executing programs stored in internal memory, ROM 12, or NVM 14, etc.

[0062] First, the processor 11 has the function of taking pictures of the front of the vehicle 10 using the camera 2.

[0063] Processor 11 initiates recording by camera 2 via camera interface 18. Upon initiation of recording, processor 11 acquires the captured images (photographs) from camera 2. Processor 11 acquires photographic images from camera 2 in real time.

[0064] In addition, the processor 11 has the function of obtaining device information.

[0065] For example, processor 11 uses antenna 3 and antenna interface 19 to obtain the current position of vehicle 10. When the current position is obtained, processor 11 generates a request for device information based on the current position. The request includes the current position. When the request is generated, processor 11 sends the generated request to host device 20 through communication unit 15.

[0066] When a request is sent to the host device 20, the processor 11 receives a response containing device information from the host device 20 via the communication unit 15.

[0067] Next, the equipment information will be explained.

[0068] The device information indicates the geodetic coordinates of the device that may be captured in the images taken by camera 2.

[0069] Figure 3 An example of a structure representing device information. For example... Figure 3 As shown, the device information is stored by establishing a correspondence between "ID", "Category", "Geometry Coordinates" and "Location Relationship with Line".

[0070] "ID" stands for Identifier used to identify the device. Here, "ID" is a numerical value.

[0071] "Category" indicates the category of the equipment. Here, "Category" is a utility pole, stop sign, or speed sign.

[0072] "Geometry coordinates" refers to the geometry coordinates of the equipment. Here, "geometry coordinates" indicates the latitude and latitude of the equipment.

[0073] "Positional relationship with the line" indicates which side of line R the device is present on in the photographic image. That is, "Positional relationship with the line" indicates whether the device is on the right or left side of line R.

[0074] In addition to the equipment information Figure 3 In addition to the structure shown, other structures may be provided as needed, or specific structures may be excluded from the device information. The structure of the device information is not limited to a specific structure.

[0075] In addition, the processor 11 has the function of obtaining line information.

[0076] The processor 11 sends a request for line information to the host device 20 via the communication unit 15. The processor 11 receives a response containing line information from the host device 20 via the communication unit 15.

[0077] In addition, the processor 11 has the function of identifying the device captured in the photographic image.

[0078] The processor 11 identifies the device from the photographed image according to the prescribed image recognition algorithm.

[0079] For example, NVM14 pre-stores dictionary information for device identification. This dictionary information could be related to the feature values ​​of the images captured of the device, or it could be information obtained through a deep learning network.

[0080] The processor 11 identifies the device based on dictionary information. That is, the processor 11 determines the coordinates (e.g., a vertex coordinate or center coordinate, etc.) of the area where the device is captured (device area).

[0081] Figure 4 This indicates that the processor 11 recognizes an example of the device's actions. Figure 4 An example of a photographic image is shown. Here, it is assumed that processor 11 identifies device regions 21 to 25.

[0082] Equipment area 21 is the area where the equipment captures images of the utility poles.

[0083] Equipment area 22 is the area where the equipment captures images of utility poles.

[0084] Equipment area 23 is the area where the speed is indicated by the image captured by the equipment.

[0085] Equipment area 24 is the area where the equipment captures the stop sign.

[0086] Equipment area 25 is the area where the equipment captures images of utility poles.

[0087] In addition, the processor 11 can further identify the device based on device information.

[0088] For example, the processor 11 can also define a search area for each device based on the "positional relationship with the line" of the device information. For example, the processor 11 can also set the search area for devices whose "positional relationship with the line" is "right" on the right side of the captured image. Alternatively, the processor 11 can also determine the position of the line through image recognition and set the search area to the right or left of the line's position.

[0089] Additionally, the processor 11 can also define the search area for each device based on the "geodetic coordinates" of the device information. For example, the processor 11 can also set the search area around the area in the photographed image that is presumed to be the location of the device based on the "geodetic coordinates" and the current position of the vehicle 10.

[0090] In addition, the processor 11 has the function of generating a transformation matrix that establishes a correspondence between the coordinates in the photographic image and the geodetic coordinates based on the coordinates and line information of the device area.

[0091] Here, the transformation matrix is ​​the matrix that transforms geodesic coordinates into coordinates in the photographic image. That is, the transformation matrix is ​​used to calculate the coordinates of an object in the photographic image that exists in a given geodesic coordinate system.

[0092] The processor 11 selects a predetermined number (e.g., 4) of device regions from the captured image. For example, the processor 11 selects device regions in such a way that the area of ​​the polygon connecting the coordinates of the device regions is maximized.

[0093] When a device region is selected, the processor 11 obtains the geodetic coordinates of the device captured in the selected device region from the device information. For example, the processor 11 obtains the geodetic information corresponding to the ID of the device captured in each of the selected device regions from the device information.

[0094] When the geodetic coordinates of the device are obtained, the processor 11 generates a transformation matrix according to the prescribed algorithm, which establishes the coordinates of each device area in the photographic image and the geodetic coordinates of the device captured in each device area.

[0095] In addition, the processor 11 has the function of generating a coordinate column of line R in the photographic image based on the transformation matrix.

[0096] When generating the transformation matrix, the processor 11 applies the transformation matrix to the geodesic coordinate column of the line R shown in the line information, and calculates the coordinate column of the line R in the photographic image. That is, the processor 11 transforms the geodesic coordinate column representing each point of the line R into the coordinate column in the photographic image.

[0097] Figure 5 Represents the coordinates of a line in a photographic image. Figure 5 In the example shown, processor 11 calculates the coordinate column 31 representing line R. Here, processor 11 calculates the coordinate columns of the right track and the left track of line R as coordinate column 31.

[0098] When coordinate column 31 is calculated, processor 11 calculates the offset used to add to coordinate column 31 to calculate the coordinates of the line based on the feature quantities of the photographed image.

[0099] For example, the processor 11 extracts the coordinates of edges in the photographic image as feature quantities of the photographic image through edge detection.

[0100] Figure 6 This represents the feature quantity detected through edge detection. For example... Figure 6 As shown, the processor 11 detects the edges at the coordinates of the lines (here, the left and right tracks) in the photographed image.

[0101] When the coordinates of an edge in a photographic image are detected, the processor 11 matches the coordinate column 31 with the feature quantity (the coordinates of the edge) and calculates an evaluation value. Here, the greater the similarity between the coordinate column 31 and the feature quantity, the higher the evaluation value.

[0102] Processor 11 adds an offset (x-coordinate and y-coordinate) to coordinate column 31 to calculate the evaluation value. Processor 11 determines the evaluation value to be the maximum offset while moving the offset in the x-axis and y-axis directions. For example, processor 11 can move the offset randomly or move the offset in the direction that increases the evaluation value.

[0103] When the evaluation value is determined to be the maximum offset, the processor 11 obtains the coordinate column with the offset added to the coordinate column 31 as the coordinate column to represent the line R in the photographic image.

[0104] In addition, the processor 11 has the function of acquiring boundary information that represents vehicle boundaries and building boundaries.

[0105] The processor 11 sends a request for boundary information to the host device 20 via the communication unit 15. The processor 11 receives a response containing the boundary information from the host device 20 via the communication unit 15.

[0106] In addition, the processor 11 has the function of setting the driving area of ​​the vehicle 10 in the photographic image based on the coordinate column and boundary information of the line R in the photographic image.

[0107] For example, processor 11 calculates the size and location of vehicle boundaries and building boundaries at each coordinate point of route R in the photographic image based on the coordinate column and geodesic coordinates of route R. Once the size and location of the vehicle boundaries and building boundaries are calculated, processor 11 sets the vehicle boundaries and building boundaries as driving areas according to their size and location.

[0108] In addition, the processor 11 has the function of detecting obstacles that hinder the movement of the vehicle 10 based on the driving area.

[0109] When a driving area is set, the processor 11 detects obstacles that hinder the driving of the vehicle 10 within the driving area. That is, the processor 11 detects obstacles that interfere with the driving area.

[0110] When an obstacle is detected, the processor 11 displays this information on the display unit 17. For example, the processor 11 may display a warning or similar information on the display unit 17.

[0111] Furthermore, the processor 11 can display different information depending on whether an obstacle is detected at the vehicle boundary or at a building boundary. For example, if an obstacle is detected at a building boundary, the processor 11 will display a notice on the display unit 17. If an obstacle is detected at the vehicle boundary, the processor 11 will display a warning on the display unit 17.

[0112] In addition, the processor 11 can also output warning sounds through a speaker or the like.

[0113] Next, an example of the operation of vehicle 10 will be explained.

[0114] Figure 7 This is a flowchart illustrating an example of the operation of vehicle 10. Here, it is assumed that vehicle 10 is traveling on route R.

[0115] First, the processor 11 of the vehicle 10 acquires a photographic image from the camera 2 (S11). When the photographic image is acquired, the processor 11 obtains device information from the host device 20 via the communication unit 15 (S12).

[0116] When device information is obtained, processor 11 obtains line information from host device 20 via communication unit (S13). When line information is obtained, processor 11 determines the device area from the photographed image (S14).

[0117] Once the equipment area is determined, the processor 11 generates a transformation matrix (S15) based on the coordinates of the determined equipment area to transform the geodesic coordinates into coordinates in the photographic image. When generating the transformation matrix, the processor 11 uses the transformation matrix to transform the geodesic coordinate column of line R shown in the line information into the coordinate column of the photographic image (S16).

[0118] When the coordinate column is transformed, the processor 11 matches the transformed coordinate column with the feature values ​​of the photographic image and calculates the coordinate column of line R in the photographic image (S17). When the coordinate column of line R in the photographic image is calculated, the processor 11 obtains the boundary information from the host device 20 through the communication unit 15 (S18).

[0119] When boundary information is obtained, the processor 11 sets a driving area in the photographic image based on the coordinate column of the line R in the photographic image and the boundary information (S19). When the driving area is set, the processor 11 detects obstacles based on the driving area (S20).

[0120] If an obstacle is detected (S20: Yes), the processor 11 will display a warning or the like on the display unit 17 (S21).

[0121] If it is determined that there is no obstacle (S20: No), or if a warning is displayed on the display unit 17 (S21), the processor 11 terminates its operation.

[0122] In addition, the processor 11 can also repeatedly execute S11 to S21 at predetermined intervals.

[0123] In addition, the device identified by the processor 11 from the photographed image may also be a tag or the like.

[0124] Alternatively, the processor 11 can overlay the coordinate column of the line R in the photographic image onto the photographic image and display it on the display unit 17. Additionally, the processor 11 can overlay the driving area onto the photographic image and display it on the display unit 17.

[0125] In addition, the processor 11 can automatically stop the vehicle 10 when an obstacle is detected. Furthermore, the processor 11 can also notify the host device 20 of the obstacle detection via the communication unit 15.

[0126] The vehicle, configured as described above, identifies equipment from photographic images. The vehicle generates a transformation matrix based on the device's geodesic coordinates and its coordinates within the photographic image. Using this transformation matrix, the vehicle transforms the geodesic coordinates of the route into its coordinates within the photographic image. As a result, even in situations where route identification through image processing is difficult, such as when the route is far away and small in the photographic image, the vehicle can effectively obtain the route's coordinates from the photographic image.

[0127] (Second Implementation)

[0128] Next, the second embodiment will be described.

[0129] The difference between the vehicle of the second embodiment and the vehicle of the first embodiment is that the route is identified from the photographic image based on the slope of the location where the equipment is located. Therefore, for other points, the same reference numerals are used and detailed descriptions are omitted.

[0130] Figure 8 This is a side view showing the route R traveled by the vehicle 10 in the second embodiment and the surrounding equipment. Additionally, Figure 9 This is a top view showing line R and its equipment.

[0131] like Figure 8 and Figure 9 As shown, the slope of line R changes at the slope change point. Here, line R closer to the slope change point is placed in the region with a slope value of Z1. Conversely, line R further in than the slope change point is placed in the region with a slope value of Z2. For example, the slope value can be expressed as per mille or an angle.

[0132] Vehicle 10 is located on track R, which is set up in an area with an inclination value of Z1. Vehicle 10 is moving from its current position toward track R, which is set up in an area with an inclination value of Z2.

[0133] Additionally, equipment (in this case, utility poles) is installed in the area with a slope value of Z1. Also, equipment (in this case, utility poles) is installed in the area with a slope value of Z2.

[0134] Additionally, the images captured by camera 2 show the line R and equipment installed in an area with an inclination value of Z1. Furthermore, the images captured by camera 2 show the line R and equipment installed in an area with an inclination value of Z2.

[0135] In addition, the route information includes the slope value of route R. That is, the route information includes the slope value at each geodetic coordinate of route R.

[0136] In addition, the equipment information also includes the slope value.

[0137] Figure 10 An example of a structure representing device information in the second embodiment. For example... Figure 10 As shown, the device information is stored by establishing a correspondence between "ID", "Category", "Geometry Coordinates" and "Slope Value".

[0138] "ID", "Category", and "Geodesic Coordinates" are as described above.

[0139] "Slope Value" indicates the slope value of the area where the equipment is installed. Here, "Slope Value" is shown as Z1 or Z2. For example... Figure 10 As shown, devices with "ID" values ​​of 1 to 4 are located in the area with a slope value of Z1. Additionally, devices with "ID" values ​​of 5 to 8 are located in the area with a slope value of Z2.

[0140] In addition, equipment information may also include its "locational relationship with the line." Equipment information, besides... Figure 10 In addition to the structure shown, other structures may be provided as needed, or specific structures may be excluded from the device information. The structure of the device information is not limited to a specific structure.

[0141] Next, the functions implemented by vehicle 10 will be explained. The functions implemented by vehicle 10 are achieved by processor 11 executing programs stored in internal memory, ROM 12, or NVM 14, etc.

[0142] In addition to the functions of the first embodiment, the vehicle 10 of the second embodiment also performs the following functions.

[0143] Processor 11 has the function of generating a transformation matrix for each slope value.

[0144] Here, the processor 11 determines the device area based on the photographed image.

[0145] The processor 11 obtains the slope value corresponding to the identified device from the device information. For example, the processor 11 obtains the slope value corresponding to the ID of the identified device from the geodetic information.

[0146] When the slope value corresponding to the device is obtained, the processor 11 sets the specified slope value (e.g., Z1).

[0147] When a slope value is set, the processor 11 determines the device region of the device corresponding to the set slope value. Once the device region is determined, the processor 11 selects a predetermined number (e.g., 4) of the determined device regions. For example, the processor 11 selects device regions in such a way that the area of ​​the polygon connecting the coordinates of the device regions is maximized.

[0148] When a device region is selected, the processor 11 obtains the geodetic coordinates of the device captured in the selected device region from the device information. For example, the processor 11 obtains the geodetic information corresponding to the ID of the device captured in each of the selected device regions from the device information.

[0149] When the geodesic coordinates of the device are obtained, the processor 11 generates a transformation matrix that establishes a correspondence between the coordinates of each device area in the photographic image and the geodesic coordinates of the device captured in each device area. The processor 11 obtains the generated transformation matrix as the transformation matrix for a set slope value (e.g., Z1). That is, the transformation matrix is ​​a matrix used to transform the geodesic coordinates of an object placed in an area with a set slope value into coordinates in the photographic image.

[0150] Processor 11 sets other slope values ​​(e.g., Z2). When other slope values ​​are set, processor 11 similarly generates a transformation matrix for the set slope value (e.g., Z2).

[0151] The processor 11 has the function of generating a coordinate column of line R in the photographic image based on the transformation matrix for each slope.

[0152] The processor 11 sets a specified slope value (e.g., Z1). When the slope value is set, the processor 11 obtains the geodetic coordinate column corresponding to the set slope value from the line information.

[0153] Once the geodesic coordinates are obtained, the processor 11 applies the transformation matrix of the set slope values ​​to the geodesic coordinates to calculate the coordinates in the photographic image. That is, the processor 11 transforms the geodesic coordinates into the coordinates in the photographic image.

[0154] When the coordinate column in the photographic image is calculated, the processor 11 sets other slope values ​​(e.g., Z2). When other slope values ​​are set, the processor 11 similarly uses the transformation matrix of the slope values ​​to transform the geodesic coordinate column corresponding to the set slope values ​​into the coordinate column in the photographic image.

[0155] Processor 11 similarly transforms the geodesic coordinate column into the coordinate column in the photographic image for each slope value.

[0156] The processor 11 merges the coordinate columns transformed by the transformed coordinates of each slope value to generate a single coordinate column. When a single coordinate column is generated, the processor 11, similar to the first embodiment, calculates the offset of the line coordinates by adding the coordinates to the coordinate column based on the feature values ​​of the photographic image.

[0157] The processor 11 obtains a coordinate column by adding an offset to the merged coordinate column and uses it as the coordinate column to represent the line R in the photographic image.

[0158] In addition, the processor 11 has the function of setting the driving area of ​​the vehicle 10 in the photographic image based on the coordinate column of the line R, the slope value of the line R and the boundary information.

[0159] For example, processor 11 calculates the size and position of vehicle boundaries and building boundaries at each coordinate point of road R in the photographic image based on the coordinate column, slope value, and geodesic coordinates of road R. Once the size and position of the vehicle boundaries and building boundaries are calculated, processor 11 sets the vehicle boundaries and building boundaries as driving areas according to their size and position.

[0160] Next, an example of the operation of vehicle 10 will be explained.

[0161] Figure 11 This is a flowchart illustrating an example of the operation of vehicle 10. Here, it is assumed that vehicle 10 is traveling on route R.

[0162] First, the processor 11 of the vehicle 10 acquires a photographic image from the camera 2 (S31). When the photographic image is acquired, the processor 11 obtains device information from the host device 20 via the communication unit 15 (S32).

[0163] When the device information is obtained, the processor 11 obtains the line information from the host device 20 through the communication unit (S33). When the line information is obtained, the processor 11 determines the device area based on the photographed image (S34).

[0164] Once the equipment area is determined, the processor 11 generates a transformation matrix for each slope value based on the coordinates of the determined equipment area (S35). After generating the transformation matrix for each slope value, the processor 11 transforms the geodesic coordinate column of the line R into the coordinate column in the photographic image based on the transformation matrix for each slope value and the line information (S36).

[0165] When the coordinate column is transformed, the processor 11 matches the transformed coordinate column with the feature values ​​of the photographic image to calculate the coordinate column of line R in the photographic image (S37). When the coordinate column of line R in the photographic image is calculated, the processor 11 obtains the boundary information from the host device 20 through the communication unit 15 (S38).

[0166] When generating boundary information, the processor 11 sets a driving area in the photographic image based on the coordinate column of the line R, the slope value of the line R, and the boundary information (S39). When the driving area is set, the processor 11 detects obstacles based on the driving area (S40).

[0167] When an obstacle is detected (S40: Yes), the processor 11 displays a warning or the like on the display unit 17 (S41).

[0168] When it is determined that there is no obstacle (S40: No), or when a warning is displayed on the display unit 17 (S41), the processor 11 ends its operation.

[0169] In addition, the processor 11 can also repeatedly execute S31 to S41 at predetermined intervals.

[0170] Additionally, the processor 11 can also obtain the slope values ​​of each point on line R independently of the line information. Furthermore, the processor 11 can also obtain the slope values ​​of the device independently of the device information.

[0171] The vehicle, configured as described above, generates a transformation matrix for each slope value of the area where the equipment is installed. Using this transformation matrix for each slope value, the vehicle transforms the geodetic coordinates of the track into the coordinates of the photographic image. As a result, even when the slope of the area where the equipment and track are installed changes, the vehicle can effectively obtain the coordinates of the track from the photographic image.

[0172] Several embodiments of the present invention have been described, but these embodiments are provided as examples and are not intended to limit the scope of the invention. These new embodiments can be implemented in various other ways, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their variations are included within the scope or spirit of the invention, and are included within the scope of the invention as described in the claims and its equivalents.

[0173] [Explanation of reference numerals in the attached figures]

[0174] 1…Housing, 2…Camera, 3…Antenna, 4…Information processing device, 10…Vehicle, 11…Processor, 12…ROM, 13…RAM, 14…NVM, 15…Communication unit, 16…Operation unit, 17…Display unit, 18…Camera interface, 19…Antenna interface, 20…Host device, 21…Equipment area, 22…Equipment area, 23…Equipment area, 24…Equipment area, 25…Equipment area, 31…Coordinate column, 100…Vehicle system.

Claims

1. An information processing apparatus comprising: an image interface that acquires a photographed image including a route and a device; a communication interface that acquires route information indicating geodetic coordinates of the route and device information indicating geodetic coordinates of the device; and a processor that performs processing of: determining a device region in which the device is photographed from the photographed image, generating a transformation matrix that establishes a correspondence between geodetic coordinates and coordinates in the photographed image based on coordinates of the device region in the photographed image and the geodetic coordinates of the device, determining coordinates of the route in the photographed image by transforming the geodetic coordinates of the route into coordinates in the photographed image based on the transformation matrix, the communication interface acquires limit information indicating a vehicle limit or a building limit of the route, the processor sets a travel region of a vehicle traveling on the route in the photographed image based on the limit information and the coordinates of the route in the photographed image. 2.The information processing apparatus according to claim 1, wherein the processor performs processing of: extracting a feature amount of the photographed image, calculating an evaluation value by matching coordinates obtained by adding an offset to the coordinates transformed from the geodetic coordinates of the route with the feature amount, determining an offset used to calculate the coordinates of the route by adding the offset to the coordinates transformed from the geodetic coordinates of the route based on the evaluation value, calculating the coordinates of the route in the photographed image by adding the offset determined to the coordinates transformed from the geodetic coordinates of the route. 3.The information processing apparatus according to claim 2, wherein the feature amount is extracted by edge detection. 4.The information processing apparatus according to claim 1, wherein the processor detects an obstacle that hinders travel of the vehicle based on the travel region. 5.The information processing apparatus according to any one of claims 1 to 3, wherein the processor performs processing of: selecting a predetermined number of the device regions from the device regions, generating the transformation matrix based on coordinates of the selected device regions. 6.The information processing apparatus according to claim 5, wherein the processor selects the device regions in a manner that an area of a polygon that links coordinates of the device regions becomes maximum. 7.The information processing apparatus according to any one of claims 1 to 3, wherein the device information includes gradient values of regions in which the device is disposed, the processor generates the transformation matrix for each of the gradient values. 8.The information processing apparatus according to any one of claims 1 to 3, wherein the device includes one of a utility pole, a speed sign, or a stop sign. 9.The information processing apparatus according to claim 4, wherein the image interface acquires the photographed image from a camera that photographs a front of the vehicle. 10.The information processing apparatus according to claim 4, wherein The antenna interface is connected to an antenna that receives a signal for positioning the vehicle, The device information indicates geodetic coordinates of the device photographed from the vehicle.

11. A program product, which is executed by a processor, The program product causes the processor to function as: a function of acquiring a photographed image including a line and a device; a function of acquiring line information indicating geodetic coordinates of the line and device information indicating geodetic coordinates of the device; a function of determining a device region in which the device is photographed from the photographed image; a function of generating a transformation matrix that establishes a correspondence between geodetic coordinates and coordinates in the photographed image, based on coordinates of the device region in the photographed image and the geodetic coordinates of the device; and a function of determining coordinates of the line in the photographed image, based on the transformation matrix that transforms the geodetic coordinates of the line into coordinates in the photographed image, The program product causes the processor to function as: a function of acquiring limit information indicating a vehicle limit or a building limit of the line; and a function of setting a travel region of a vehicle traveling on the line in the photographed image, based on the limit information and the coordinates of the line in the photographed image. ​

Images