Location management system and location management method

Abstract

To obtain highly accurate position information of a moving object at a low cost.SOLUTION: A position management system according to an embodiment acquires a photographed image from a stationary camera, detects a moving object from the photographed image, and specifies actual position information of the moving object in the photographed image on the basis of position correspondence information in which the actual position information is associated in advance with each part of the photographed image. Also, the position management system acquires position information of the moving object based on GPS or GNSS from the moving object, acquires predetermined feature information about the moving object from the moving object or an external device, and extracts predetermined feature information of the moving object from the photographed image. Also, the position management system associates the moving object appearing in the photographed image with a moving object that transmitted the position information acquired by a position information acquisition unit on the basis of the actual position information specified by a specifying unit, the position information acquired by the position information acquisition unit, the predetermined feature information acquired by a feature information acquisition unit, and the predetermined feature information extracted by an extraction unit.SELECTED DRAWING: Figure 12

Description

Technical Field

[0001] Embodiments of the present invention relate to a location management system and a location management method.

Background Art

[0002] In recent years, for example, in order to realize a dynamic map used for MaaS (Mobility as a Service), accurate measurement and management of position information regarding moving objects such as vehicles are necessary. When obtaining the position information of a vehicle, for example, GNSS (Global Navigation Satellite System) can be used to obtain more accurate position information than GPS (Global Positioning System).

[0003] However, even when using GNSS, there are problems such as a large error in position information in an area where high-rise buildings are densely packed. Therefore, in order to realize advanced autonomous driving and the like, it is necessary to reduce the position measurement error. As a solution, for example, there is a method of mounting a high-precision positioning device on a vehicle.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Patent Document 3

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, the above-mentioned high-precision positioning device has a problem of being expensive.

[0006] Therefore, the problem to be solved by the present invention is to provide a position management system and a position management method capable of obtaining high-precision position information of an object at low cost.

Means for Solving the Problem

[0007] The position management system according to the embodiment includes: a captured image acquisition unit that acquires a captured image from a stationary camera that captures a predetermined area where a moving object moves; a detection unit that detects the moving object from the captured image; a specifying unit that specifies the actual position information of the moving object shown in the captured image based on position correspondence information in which actual position information is associated in advance with each part of the captured image by the stationary camera; a position information acquisition unit that acquires the position information of the moving object based on GPS or GNSS from the moving object moving in the predetermined area; a feature information acquisition unit that acquires predetermined feature information regarding the moving object from the moving object or an external device; an extraction unit that extracts the predetermined feature information of the moving object from the captured image; and a control unit that associates the moving object shown in the captured image with the moving object that transmitted the position information acquired by the position information acquisition unit based on the actual position information specified by the specifying unit, the position information acquired by the position information acquisition unit, the predetermined feature information acquired by the feature information acquisition unit, and the predetermined feature information extracted by the extraction unit.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

[0009] Hereinafter, embodiments of the position management system and the position management method of the present invention will be described with reference to the drawings. Hereinafter, a vehicle will be described as an example of a moving object.

[0010] First, with reference to FIG. 1, a rough difference between the embodiment and the comparative example (prior art) will be described. FIG. 1 is a schematic diagram for explaining a rough difference between the embodiment and the comparative example (prior art). In the comparative example (prior art), it is assumed that a dynamic map is created using position information obtained by a high-precision positioning device mounted on a vehicle. In this case, although high-precision position information of the vehicle can be obtained in the entire area where the vehicle travels, there is a problem that the high-precision positioning device is expensive.

[0011] On the one hand, in this embodiment, it is assumed that a dynamic map is created using the position information by the proposed model described below. In the proposed model, roads are classified into two types of areas, and the level of vehicle position measurement accuracy is divided. One is an area where high-precision position measurement is required, such as an intersection, and it is the application target of the proposed model. The other is an area where high-precision position measurement is not required, such as a straight road, and it is not the application target of the proposed model, and the position information of the GPS sensor owned by each vehicle is used. According to this method, high-precision vehicle position information can be obtained at low cost in the high-precision position measurement area. This will be described in detail below.

[0012] FIG. 2 is an overall configuration diagram of the position management system S of the embodiment. The position management system S includes a position management server 1, a PC (Personal Computer) 2, a fixed camera 3, and a map management server 4 (management server).

[0013] The fixed camera 3 photographs a predetermined area (for example, a road intersection, etc.) where the vehicle 5 (hereinafter also referred to as "vehicle" without a symbol) moves. More specifically, the fixed camera 3 is installed, for example, at a high position in a main place on the road and has an angle of view that looks down on the road from above. The range of the field of view of this fixed camera 3 becomes the high-precision position measurement area. In addition, the fixed camera 3 is installed at various places on the road, and one PC 2 is installed for one fixed camera 3.

[0014] Here, with reference to FIG. 3, an overview of the processing by the PC 2 of the embodiment will be described. FIG. 3 is a diagram for explaining an overview of the processing by the PC 2 of the embodiment. FIG. 3(a) is an example of an image obtained by the fixed camera 3 (hereinafter also referred to as a fixed camera image).

[0015] As shown in FIG. 3(a), the PC2 detects a vehicle from a stationary camera image. In FIG. 3(b), the detected vehicle is surrounded by a rectangle. Then, the PC2 identifies the position information for the detected vehicle. FIG. 3(c) schematically shows that the center position P10 of the bottom surface in the wireframe L applied to the vehicle is identified as the position information of the vehicle using the wireframe method.

[0016] Returning to FIG. 2, the PC2 includes a processing unit 21, a storage unit 22, an input unit 23, a display unit 24, and a communication unit 25.

[0017] The storage unit 22 is a storage device such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive), and stores various information. The storage unit 22 stores, for example, a position DB (Data Base) 221 (position correspondence information), a conversion table 222 (position correspondence information), first vehicle information 223, and a vehicle type discrimination table 224.

[0018] The position DB 221 is a DB that defines the correspondence between a stationary camera image and a satellite image (or an aerial image, an example of an overhead image). Here, with reference to FIG. 4, the correspondence between the stationary camera image and the satellite image in the embodiment will be described.

[0019] FIG. 4 is a diagram for explaining the correspondence between a stationary camera image and a satellite image in the embodiment. In the position DB 221, for the movable range of the vehicle 5, each pixel in the stationary camera image shown in FIG. 4(a) is associated with each pixel in the satellite image shown in FIG. 4(b). Therefore, by referring to this position DB 221, the pixel in the satellite image corresponding to the pixel in the stationary camera image can be identified. Further, this position DB 221 may be realized by a projective transformation matrix generated based on the coordinates of a plurality of sets of corresponding points between the stationary camera image and the satellite image.

[0020] Returning to FIG. 2, the conversion table 222 is a table in which the coordinates of the pixels of the satellite image are associated with the position information of the satellite image. Here, with reference to FIG. 5, the conversion table 222 of the embodiment will be described.

[0021] FIG. 5 is a diagram showing the conversion table 222 of the embodiment. In the conversion table 222, the coordinates (X coordinate, Y coordinate) of the pixels of the satellite image are associated with the high-precision position information (latitude, longitude, altitude) of the satellite image.

[0022] Returning to FIG. 2, the first vehicle information 223 is information on the vehicle detected based on the stationary camera image. The first vehicle information 223 includes, for example, the following information (1) to (11). (1) Shooting date and time (2) Frame number of the stationary camera image (3) Coordinates of the representative points (for example, upper left and lower right) of the rectangle surrounding the vehicle (4) Bottom center coordinates of the wireframe (5) Vehicle type (ordinary car / truck / bus, etc.) (6) Vehicle body color information (for example, RGB (Red, Green, Blue) histogram) (7) Tracking ID (Identifier) (when tracking vehicles in time series) (8) Position information (latitude, longitude, altitude) (9) Direction of the vehicle (10) Travel direction (traveling direction) (11) Lane information of the vehicle in motion

[0023] Among (1) to (11), (1) and (2) are obtained from the stationary camera image. (3) to (11) are obtained by the processing of the processing unit 21.

[0024] The processing unit 21 controls the overall operation of the PC 2 and realizes various functions of the PC 2. The processing unit 21 includes, for example, a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory). The CPU comprehensively controls the operation of the PC 2. The ROM is a storage medium that stores various programs and data. The RAM is a storage medium for temporarily storing various programs and rewriting various data. The CPU executes programs stored in the ROM, the storage unit 22, etc. using the RAM as a work area (working area). The processing unit 21 includes, as functional units, an acquisition unit 211, a detection unit 212, an identification unit 213, an extraction unit 214, and a control unit 215.

[0025] The acquisition unit 211 acquires various information from external devices (for example, the stationary camera 3, the location management server 1, etc.). The acquisition unit 211 functions, for example, as a captured image acquisition unit that acquires a captured image from the stationary camera 3.

[0026] The detection unit 212 detects vehicles from the stationary camera images based on a method such as DNN (Deep Neural Network).

[0027] The identification unit 213 identifies the actual location information of the vehicle based on, for example, the center position of the bottom surface of the wireframe applied to the vehicle shown in the stationary camera image and the position in the satellite image corresponding thereto based on the location DB 221 and the conversion table 222.

[0028] The extraction unit 214 extracts predetermined feature information of the vehicle from the captured image. The predetermined feature information is, for example, vehicle type information indicating the type of the vehicle. In that case, the extraction unit 214 extracts the vehicle type information of the vehicle from the captured image. When the extraction unit 214 extracts the vehicle type information of the vehicle from the captured image, it estimates the posture of the vehicle 5 shown in the captured image, represents the vehicle as a wireframe of a rectangular parallelepiped based on the posture, and calculates the appearance features, vehicle width, vehicle length, and vehicle height based on the wireframe. Then, the extraction unit 214 extracts the vehicle type information of the vehicle based on the calculated appearance features, vehicle width, vehicle length, vehicle height, and a vehicle type discrimination table 224 (vehicle type correspondence information) in which the appearance features, vehicle width, vehicle length, vehicle height, and vehicle type information are associated in advance.

[0029] Here, the vehicle type determination and the like will be described with reference to FIGS. 6, 7, etc. First, with reference to FIGS. 3(c) and 6, an example of a method for calculating the vehicle height will be described.

[0030] FIG. 6 is a diagram for explaining an example of a method for calculating the vehicle height in the embodiment. The coordinates of the four points (P1 to P4 in FIG. 3(c)) at the corners of the bottom surface of the wireframe are projected and transformed onto the map. Next, the GPS coordinates of the four points after the projective transformation are obtained, and the vehicle width (the length between P1 and P3) and the vehicle length (the length between P1 and P2) are calculated by converting the difference in coordinates into a distance. For example, these processes are repeated for the wireframes of a plurality of captured images of the same vehicle, and an average is obtained.

[0031] Next, a method for calculating the vehicle height will be described. First, P1 and P2, which are the two ends of the vehicle length on the bottom surface of the wireframe, are selected.

[0032] Next, P1 and P1h, which are the two ends of the vehicle height in the wireframe, are selected. Next, a coordinate N pixels to the left of the screen x coordinate of P1 is set as P1a(x1 - N, y1). Also, a coordinate N pixels to the right of the screen x coordinate of P1 is set as P1b(x1 + N, y1).

[0033] Similarly, let the coordinate N pixels to the left of the screen x - coordinate of P2 be P2a(x2 - N, y2). Also, let the coordinate N pixels to the right of the screen x - coordinate of P1 be P2b(x2 + N, y2).

[0034] Next, perform projective transformation on the screen coordinates of P1a and P1b to calculate the latitude and longitude. Next, since the approximate length between pixels (taking into account points that vary depending on the position on the screen) can be calculated from the relationship between the calculated values of the vehicle width and length and the number of pixels in the image, use that value to obtain the distance 1 between P1a and P1b. For P1, the distance 11 between pixels in the x - coordinate can be calculated as distance 1÷2N.

[0035] Similarly, for P2, the distance 12 between pixels in the x - coordinate can be calculated as (distance 2 between P2a and P2b)÷2N.

[0036] The vehicle height is (the average of distance 11 and distance 12)×(y - coordinate of P1 - y - coordinate of P1h) (distance D in Figure 6). Also, as shown in Figure 6, correction based on the viewing angle α (which varies depending on the y - coordinate of the screen) of the stationary camera 3 is performed according to the following formula. Actual vehicle height H = distance D÷cos(viewing angle α)

[0037] Next, referring to Figure 7, an example of the vehicle type discrimination table 224 in the embodiment will be described. Figure 7 is a diagram showing an example of the vehicle type discrimination table 224 in the embodiment. The vehicle type of the vehicle detected by the stationary camera 3 is estimated using information (21) and information (22).

[0038] Information (21) is vehicle type classification based on appearance features using DL / DNN. Information (22) is information on vehicle width, length, and height based on the results of vehicle dimension measurement using wireframe. Using only information (21), it is difficult to distinguish vehicles with similar appearances but different sizes. However, by further using information (22), such distinction becomes possible.

[0039] In addition, information on vehicle width, vehicle length, and vehicle height for each vehicle type is defined in advance. By doing this, using the vehicle type discrimination table 224, information (21), and information (22), highly accurate estimation of the vehicle type becomes possible.

[0040] In the case where the determination result becomes unstable with this method, for example, give priority to information (22) over information (21). Also, when any of the measured values of vehicle width, vehicle length, and vehicle height in information (22) do not fall within the range of the standard dimensions of each vehicle type defined in the vehicle type discrimination table 224, for example, select the vehicle type with the smallest error between the three measured values and the standard dimensions.

[0041] Returning to FIG. 2, the extraction unit 214 extracts, for example, the body color information of the vehicle 5 from the captured image. When the extraction unit 214 extracts the body color information of the vehicle 5 from the captured image, it estimates the posture of the vehicle 5 shown in the captured image, represents the vehicle as a wireframe of a rectangular parallelepiped based on the posture, and extracts two representative colors as the body color information by clustering the color information based on the crop image, which is an image of the part of the vehicle 5 inside the wireframe.

[0042] Here, referring to FIG. 8, an example of the vehicle body color (color of the vehicle 5) determination method in the embodiment will be described. FIG. 8 is a diagram for explaining an example of the vehicle body color determination method in the embodiment. The vehicle body color is determined by the following processes (31) to (35).

[0043] First, in process (31), a crop image, which is an image of the part of the vehicle 5 inside the wireframe in the captured image, is extracted.

[0044] Next, in process (32), color information is obtained from the crop image using a clustering method. That is, since the crop image contains multiple types of colors with slightly different brightness and color tones, the color information is extracted and grouped into 5 to 7 colors by clustering similar colors together.

[0045] Next, in process (33), the color information is represented by a histogram (it may be normalized). Next, in process (34), the black component of the vehicle's tire is removed. For example, when the vehicle body is black, a certain percentage (e.g., 10%) of the black component is removed.

[0046] Next, in process (35), based on the color histogram, the vehicle body color is represented by two representative colors. For example, the color with the highest frequency in the color histogram is taken as the main body color, and the color with the second highest frequency in the color histogram or the color that characterizes the vehicle (a color not present in surrounding vehicles) is taken as the auxiliary color. Note that the main body color and the auxiliary color may be regarded as two representative colors without distinction.

[0047] In this way, the vehicle body color can be determined. Note that since the determination result of the vehicle body color may change depending on the direction of the vehicle, the degree of light reflection of the glass part, etc., for example, a majority vote of the determination results using multiple frames of images may be taken, but it is not limited to this.

[0048] Next, with reference to FIG. 9, an example of the vehicle body color determination result in the embodiment will be described. FIG. 9 is a diagram showing an example of the vehicle body color determination result in the embodiment. For the captured image shown on the left, the vehicle body color determination was performed. Regarding the six vehicles detected from the captured image, it was possible to determine the main body colors as white, blue, black, blue, black, and green in order from the top.

[0049] Returning to FIG. 2, the extraction unit 214 extracts, for example, the traveling direction information of the vehicle from a plurality of captured images in time series. Also, the extraction unit 214 extracts, for example, the lane information on which the vehicle is traveling from the captured image.

[0050] The control unit 215 executes various information processes. Also, the control unit 215 appropriately generates information that is not generated by the specific unit 213 and the extraction unit 214 among the above (3) to (11).

[0051] The input unit 23 receives the user's operation on the PC2. The input unit 23 is, for example, an input device such as a keyboard or a mouse.

[0052] The display unit 24 displays various types of information. The display unit 24 is, for example, a liquid crystal display device (LCD (Liquid Crystal Display)), an organic EL (Electro-Luminescence) display device, or the like.

[0053] The communication unit 25 is a communication interface for communicating with external devices (for example, the stationary camera 3, the location management server 1, etc.).

[0054] Next, the location management server 1 will be described. The location management server 1 is a computer device and includes a processing unit 11, a storage unit 12, an input unit 13, a display unit 14, and a communication unit 15.

[0055] The storage unit 12 is a storage device such as an HDD or an SSD and stores various types of information. The storage unit 12 stores, for example, first vehicle information 121, second vehicle information 122, and correspondence information 123.

[0056] The first vehicle information 121 is the same as the first vehicle information 223 in the PC2.

[0057] The second vehicle information 122 is the same as the second vehicle information 51 in the vehicle 5. The second vehicle information 51 includes, for example, the information in the following (16) to (18). (16) GPS date and time (17) GPS location information (latitude, longitude, altitude) (18) ETC registration information (in-vehicle unit number, registration number / vehicle number, presence / absence information of towing device, etc.)

[0058] The correspondence information 123 is information that associates the first vehicle information 121 and the second vehicle information 122 for the same vehicle 5.

[0059] The processing unit 11 controls the overall operation of the location management server 1 and realizes various functions of the location management server 1. The processing unit 11 includes, for example, a CPU, a ROM, and a RAM. The CPU comprehensively controls the operation of the location management server 1. The ROM is a storage medium that stores various programs and data. The RAM is a storage medium for temporarily storing various programs and rewriting various data. The CPU executes the programs stored in the ROM, the storage unit 12, etc. using the RAM as a work area (working area). The processing unit 11 includes, as functional units, an acquisition unit 111 and a control unit 112.

[0060] The acquisition unit 111 acquires various information from external devices (for example, the PC 2, the map management server 4, etc.). The acquisition unit 111, for example, acquires the first vehicle information 223 from the PC 2 and stores it in the storage unit 12 as the first vehicle information 121. Also, the acquisition unit 111, for example, acquires the second vehicle information 51 from the vehicle 5 and stores it in the storage unit 12 as the second vehicle information 122.

[0061] Also, the acquisition unit 111 functions as a position information acquisition unit that acquires the position information of the vehicle 5 based on GPS or GNSS from the vehicle 5 moving in a predetermined area.

[0062] Also, the acquisition unit 111 functions as a feature information acquisition unit that acquires predetermined feature information (such as vehicle type information, vehicle body color information, traveling direction information, lane information, etc.) regarding the vehicle 5 from the vehicle 5 or the external device 6. The external device 6 is a computer device that manages feature information.

[0063] The control unit 112 executes various information processes. The control unit 112, for example, based on the actual position information of the vehicle 5, the position information acquired by the position information acquisition unit (acquisition unit 111), the predetermined feature information acquired by the feature information acquisition unit (acquisition unit 111), and the predetermined feature information extracted by the extraction unit 214, associates the vehicle 5 shown in the captured image with the vehicle 5 that transmitted the position information acquired by the position information acquisition unit (acquisition unit 111). The associated information is stored in the correspondence information 123.

[0064] After performing the above-mentioned association, the control unit 112 transmits information such as "(8) position information (latitude, longitude, altitude)" in the first vehicle information 121 to the map management server 4, for example. As a result, the map management server 4 can update the dynamic map 41 using the high-precision "(8) position information (latitude, longitude, altitude)" as auxiliary information.

[0065] In addition, the control unit 112 transmits information such as "(8) position information (latitude, longitude, altitude)" in the first vehicle information 121 to the corresponding vehicle 5, for example. As a result, the vehicle 5 can utilize the high-precision "(8) position information (latitude, longitude, altitude)".

[0066] The input unit 13 receives a user's operation on the position management server 1. The input unit 13 is, for example, an input device such as a keyboard or a mouse.

[0067] The display unit 14 displays various information. The display unit 14 is, for example, a liquid crystal display device (LCD), an organic EL display device, or the like.

[0068] The communication unit 15 is a communication interface for communicating with external devices (for example, the PC 2, the map management server 4, etc.).

[0069] Next, with reference to FIG. 10, the processing by the PC 2 of the embodiment will be described. FIG. 10 is a flowchart showing the processing by the PC 2 of the embodiment. First, in step S1, the acquisition unit 211 acquires a stationary camera image from the stationary camera 3.

[0070] Next, in step S2, the detection unit 212 detects the vehicle 5 (moving object) shown in the stationary camera image based on a method such as DNN.

[0071] Next, in step S3, the specifying unit 213 specifies the position information of the vehicle 5 (moving object). Here, with reference to FIG. 11, the details of the process of step S3 in FIG. 10 will be described.

[0072] FIG. 11 is a flowchart showing the details of the process of step S3 in FIG. 10. In step S31, the specifying unit 213 refers to the position DB 221 and converts the stationary camera image into a satellite image in terms of coordinates (FIG. 4).

[0073] Next, in step S32, the specifying unit 213 calculates the bottom center coordinates of the wireframe applied to the vehicle 5 (moving object) in the satellite image (FIG. 3(c)). Next, in step S33, the specifying unit 213 refers to the conversion table 222 and converts the bottom center coordinates of the vehicle in the satellite image into position information (latitude, longitude, altitude).

[0074] Returning to FIG. 10, after step S3, in step S4, the control unit 215 generates the first vehicle information 223. Next, in step S5, the control unit 215 transmits the first vehicle information 223 to the position management server 1.

[0075] Next, with reference to FIG. 12, the process by the position management server 1 of the embodiment will be described. FIG. 12 is a flowchart showing the process by the position management server 1 of the embodiment. First, in step S41, the acquisition unit 111 acquires the first vehicle information 223 from the PC 2 and stores it in the storage unit 12 as the first vehicle information 121.

[0076] Next, in step S42, the acquisition unit 111 acquires the second vehicle information 51 from the vehicle 5 and stores it in the storage unit 12 as the second vehicle information 122.

[0077] Next, in step S43, the control unit 112 associates the first vehicle information 121 with the second vehicle information 122 for the same vehicle 5, and stores the processing result in the storage unit 12 as the correspondence information 123. Here, with reference to FIG. 13, the details of the processing in step S43 of FIG. 12 will be described.

[0078] FIG. 13 is a flowchart showing the details of the processing in step S43 of FIG. 12. In step S431, the control unit 112 narrows down by position (including the lane). For example, the association is performed using the (8) position information (latitude, longitude, altitude) of the first vehicle information 121 and the (17) GPS position information of the second vehicle information 122. This association is performed using the position information in the map coordinate system (latitude, longitude), but it may be difficult due to poor accuracy of the (17) GPS position information or the presence of multiple vehicles 5 on the road.

[0079] Next, in step S432, the control unit 112 narrows down by the traveling direction and speed of the vehicle 5. Regarding the traveling direction, for example, straight-ahead vehicles are associated with each other, and left-turning vehicles are associated with each other. Next, in step S433, the control unit 112 narrows down by vehicle type and vehicle body color. Regarding the vehicle body color, for example, the representative two colors are used for the association.

[0080] Note that the order of the processes in steps S431 to S433 is not limited to this. For example, the order of steps S432 and S433 may be reversed, or if the association fails in the first pass of the processes in steps S431 to S433, the second pass of the processes may be performed. Also, instead of using each element for narrowing down in order, they may be used simultaneously in parallel, or elements with a high probability of narrowing down may be preferentially used. Also, instead of associating the vehicles 5 one by one, they may be associated in groups of about three or four.

[0081] Returning to FIG. 12, after step S43, in step S44, the control unit 112 transmits information such as the "(8) position information (latitude, longitude, altitude)" in the first vehicle information 121 to the map management server 4.

[0082] Next, in step S45, the control unit 112 transmits information such as "(8) position information (latitude, longitude, altitude)" in the first vehicle information 121 to the corresponding vehicle 5.

[0083] Next, referring to FIG. 14, other processes by the position management server 1 of the embodiment will be described. FIG. 14 is a flowchart showing other processes by the position management server 1 of the embodiment. It is conceivable that the number of vehicles recognized based on the first vehicle information 121 is different from the number of vehicles recognized based on the second vehicle information 122. Here, it is assumed that the number of vehicles recognized based on the second vehicle information 122 is correct.

[0084] In that case, due to bad conditions such as at night or in bad weather, the detection accuracy of vehicles based on the stationary camera images may decrease. Specifically, it is conceivable that the number of vehicle rectangles detected from the stationary camera images may be less than or more than the number of vehicles traveling on the road.

[0085] When the number of vehicle rectangles is small, it means that the vehicle detection conditions are too strict. Also, when the number of vehicle rectangles is large, it means that the vehicle detection conditions are too loose. Therefore, the process of FIG. 14 is performed.

[0086] First, in step S51, the control unit 112 calculates a second vehicle number (N2), which is the number of vehicles, using the second vehicle information 122.

[0087] Next, in step S52, the control unit 112 calculates a first vehicle number (N1), which is the number of vehicles, using the first vehicle information 121.

[0088] Next, in step S53, the control unit 112 determines whether (N2 - N1) is greater than a first threshold value (for example, about 3 to 5). If Yes, it proceeds to step S54. If No, it proceeds to step S55.

[0089] In step S54, the control unit 112 performs dictionary switching (such as switching between day and night models, switching between weather models, etc.) and parameter adjustment in the DNN used by the detection unit 212 of PC2 for vehicle detection in the stationary camera image so that N1 increases. After step S54, the process returns to step S52.

[0090] In step S55, the control unit 112 determines whether (N1 - N2) is greater than a second threshold value (for example, about 3 to 5, etc.). If Yes, the process proceeds to step S56. If No, the process ends.

[0091] In step S56, the control unit 112 performs dictionary switching and parameter adjustment in the DNN used by the detection unit 212 of PC2 for vehicle detection in the stationary camera image so that N1 decreases. After step S56, the process returns to step S52. In this way, by performing dictionary switching and parameter adjustment as necessary, the numbers of N1 and N2 can be made closer, and the position information of the vehicle can be appropriately specified.

[0092] In this way, according to the position management system S of the present embodiment, in addition to the position information of the vehicle specified from the stationary camera image and the GPS position information acquired from the vehicle, by using predetermined feature information (such as vehicle type information, vehicle body color information, traveling direction information, lane information, etc.), high-precision position information of the moving object (vehicle) can be obtained at low cost.

[0093] Also, when using vehicle type information, high-precision association can be performed by using the method described with reference to FIG. 7.

[0094] Also, when using vehicle body color information, high-precision association can be performed by using the method described with reference to FIG. 8.

[0095] Also, by transmitting the position information of the vehicle to the map management server 4, the map management server 4 can appropriately update the dynamic map 41 using this high-precision position information as auxiliary information.

[0096] Also, by transmitting the position information of the vehicle to the corresponding vehicle 5, vehicle 5 can effectively utilize this highly accurate position information for autonomous driving (such as automatic driving).

[0097] The programs executed by the position management server 1 and the PC 2 of the embodiment have a module configuration including the above-described respective functional units. As actual hardware, the CPU (processor) reads the program from the above ROM and executes it, so that the above respective functional units are loaded onto the main storage device and are generated on the main storage device.

[0098] The program is provided by being recorded on a computer-readable recording medium such as a CD-ROM, a flexible disk, a CD-R, a DVD, etc. in an installable format or an executable format file.

[0099] Alternatively, the program may be stored on a computer connected to a network such as the Internet and provided by being downloaded via the network. Also, the program may be configured to be provided or distributed via a network such as the Internet. Also, the program may be configured to be provided by being pre-embedded in a ROM or the like.

[0100] As described above, the embodiments of the present invention have been described. However, the above embodiments are merely examples and are not intended to limit the scope of the invention. The above embodiments can be implemented in various forms, and various omissions, replacements, and changes can be made without departing from the gist of the invention. The above embodiments are included in the scope and gist of the invention and are included in the invention described in the claims and its equivalent scope.

[0101] For example, the moving object is not limited to a vehicle, and may be, for example, other moving objects such as a moving body (transport vehicle, mobile robot, etc.) in a factory.

[0102] Also, the functions of the location management server 1 and the PC 2 may be realized by one computer device, or may be realized by three or more computer devices.

[0103] Also, when there is distortion in the stationary camera image, the image may be corrected such that the white line of the crosswalk in the image becomes a straight line. Also, for satellite images, when there is distortion, the image may be corrected in the same manner.

[0104] Also, the second vehicle information may be generated by other methods such as GNSS instead of GPS.

[0105] Also, for one shooting area, a plurality of stationary cameras 3 may be installed and a plurality of stationary camera images may be used.

Explanation of Signs

[0106] 1…Location management server, 2…PC, 3…Stationary camera, 4…Map management server, 5…Vehicle, 11…Processing unit, 12…Storage unit, 13…Input unit, 14…Display unit, 15…Communication unit, 21…Processing unit, 22…Storage unit, 23…Input unit, 24…Display unit, 25…Communication unit, 41…Dynamic map, 51…Second vehicle information, 111…Acquisition unit, 112 Control unit, 121…First vehicle information, 122…Second vehicle information, 123…Corresponding information, 211…Acquisition unit, 212…Detection unit, 213…Specification unit, 214…Extraction unit, 215…Control unit, 221…Location DB, 222…Conversion table, 223…First vehicle information, 224…Vehicle type discrimination table, S…Location management system

Claims

An imaging image acquisition unit that acquires an imaging image from a stationary camera that images a predetermined area where the vehicle moves, A detection unit that detects the vehicle from the imaging image, A specifying unit that specifies the actual position information of the vehicle shown in the imaging image based on position correspondence information in which actual position information is associated in advance with each part of the imaging image taken by the stationary camera, A position information acquisition unit that acquires the position information of the vehicle based on GPS (Global Positioning System) or GNSS (Global Navigation Satellite System) from the vehicle moving in the predetermined area, A feature information acquisition unit that acquires body color information about the vehicle from the vehicle or an external device, When extracting the body color information of the vehicle from the imaging image, estimating the posture of the vehicle shown in the imaging image, representing the vehicle by a wireframe of a rectangular parallelepiped based on the posture, and clustering the color information based on a crop image that is an image of a part of the vehicle inside the wireframe to extract two representative colors as the body color information, A control unit that associates the vehicle shown in the imaging image with the vehicle that transmitted the position information acquired by the position information acquisition unit based on the actual position information specified by the specifying unit, the position information acquired by the position information acquisition unit, the body color information acquired by the feature information acquisition unit, and the body color information extracted by the extraction unit, Comprising A position management system. An imaging image acquisition unit that acquires an imaging image from a stationary camera that images a predetermined area where the vehicle moves, A detection unit that detects the vehicle from the imaging image, A specifying unit that specifies the actual position information of the vehicle shown in the imaging image based on position correspondence information in which actual position information is associated in advance with each part of the imaging image taken by the stationary camera, A position information acquisition unit that acquires the position information of the vehicle based on GPS (Global Positioning System) or GNSS (Global Navigation Satellite System) from the vehicle moving in the predetermined area, A feature information acquisition unit that acquires traveling direction information about the vehicle from the vehicle or an external device, An extraction unit that extracts the traveling direction information of the vehicle from a plurality of the captured images in time series; A control unit that associates the vehicle shown in the captured image with the vehicle that transmitted the position information acquired by the position information acquisition unit based on the actual position information specified by the specifying unit, the position information acquired by the position information acquisition unit, the traveling direction information acquired by the feature information acquisition unit, and the traveling direction information extracted by the extraction unit; Comprising A position management system.

3. A captured image acquisition unit that acquires a captured image from a stationary camera that captures a predetermined area where the vehicle moves; A detection unit that detects the vehicle from the captured image; A specifying unit that specifies the actual position information of the vehicle shown in the captured image based on position correspondence information in which actual position information is associated in advance with each part of the captured image by the stationary camera; A position information acquisition unit that acquires the position information of the vehicle based on GPS (Global Positioning System) or GNSS (Global Navigation Satellite System) from the vehicle moving in the predetermined area; A feature information acquisition unit that acquires lane information regarding the vehicle from the vehicle or an external device; An extraction unit that extracts lane information on which the vehicle travels from the captured image; A control unit that associates the vehicle shown in the captured image with the vehicle that transmitted the position information acquired by the position information acquisition unit based on the actual position information specified by the specifying unit, the position information acquired by the position information acquisition unit, the lane information acquired by the feature information acquisition unit, and the lane information extracted by the extraction unit; Comprising A position management system.

4. The position management system according to any one of claims 1 to 3, wherein the control unit transmits the position information of the vehicle to the vehicle and / or a management server of a dynamic map.

5. A captured image acquisition step of acquiring a captured image from a stationary camera that captures a predetermined area where the vehicle moves; A detection step of detecting the vehicle from the captured image; A specifying step of specifying the actual position information of the vehicle shown in the captured image based on position correspondence information in which actual position information is associated in advance with each part of the captured image by the stationary camera; A position information acquisition step of acquiring position information of the vehicle based on GPS or GNSS from the vehicle moving in the predetermined area; A feature information acquisition step of acquiring vehicle body color information regarding the vehicle from the vehicle or an external device; When extracting the vehicle body color information from the captured image, estimating the posture of the vehicle shown in the captured image, representing the vehicle by a wireframe of a rectangular parallelepiped based on the posture, and clustering the color information based on a crop image which is an image of the portion of the vehicle inside the wireframe to extract two representative colors as the vehicle body color information; A control step of associating the vehicle shown in the captured image with the vehicle that transmitted the position information acquired in the position information acquisition step based on the actual position information specified in the specifying step, the position information acquired in the position information acquisition step, the vehicle body color information acquired in the feature information acquisition step, and the vehicle body color information extracted in the extraction step; including a position management method. A captured image acquisition step of acquiring a captured image from a fixed camera that captures a predetermined area where the vehicle moves; A detection step of detecting the vehicle from the captured image; A specifying step of specifying the actual position information of the vehicle shown in the captured image based on position correspondence information in which actual position information is associated in advance with each part of the captured image by the fixed camera; A position information acquisition step of acquiring position information of the vehicle based on GPS or GNSS from the vehicle moving in the predetermined area; A feature information acquisition step of acquiring traveling direction information regarding the vehicle from the vehicle or an external device; An extraction step of extracting the traveling direction information of the vehicle from a plurality of the captured images in time series; A control step of associating the vehicle shown in the captured image with the vehicle that transmitted the position information acquired in the position information acquisition step based on the actual position information specified in the specifying step, the position information acquired in the position information acquisition step, the traveling direction information acquired in the feature information acquisition step, and the traveling direction information extracted in the extraction step; including a position management method. A photographing image acquisition step of acquiring a photographed image from a fixed camera that photographs a predetermined area where the vehicle moves; A detection step of detecting the vehicle from the photographed image; An identification step of identifying the actual position information of the vehicle shown in the photographed image based on position correspondence information in which actual position information is associated in advance with each part of the photographed image by the fixed camera; A position information acquisition step of acquiring the position information of the vehicle based on GPS or GNSS from the vehicle moving in the predetermined area; A feature information acquisition step of acquiring lane information regarding the vehicle from the vehicle or an external device; An extraction step of extracting lane information on which the vehicle travels from the photographed image; A control step of associating the vehicle shown in the photographed image with the vehicle that transmitted the position information acquired in the position information acquisition step based on the actual position information identified in the identification step, the position information acquired in the position information acquisition step, the lane information acquired in the feature information acquisition step, and the lane information extracted in the extraction step; Including A position management method.

Images