Information Processing Method

Abstract

To provide an information processing method that enhances applicability of information on a target.SOLUTION: An information processing method is executed by at least one processor 11b, 52b so as to output information on a target in a photographing area based on an image of the photographing area which is obtained by photographing by an infrastructure camera 2. The information processing method includes: obtaining a probability distribution of a pixel error for each distance range from the infrastructure camera 2 to the photographing area in the image; obtaining an upper-limit error in an allowable probability range in the probability distribution, for each distance range; and outputting, as the information on the target, both a position of the target in a map coordinate system and an estimated position error of the target which is obtained by converting the upper-limit error into a range in the map coordinate system.SELECTED DRAWING: Figure 14

Description

[Technical Field]

[0001] The disclosure of this specification relates to a technology for outputting information about a target based on an image captured by an imaging device and utilizing the information about the target. [Background technology]

[0002] Patent Document 1 discloses a technology for outputting target information based on an image of a photographed area photographed by a photographing device. Specifically, a target is detected based on position information expressed in an individual coordinate system specific to the photographing device, and the position of the target detected based on the position expressed in the individual coordinate system is converted to a position expressed in a common coordinate system and output. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-36143 Summary of the Invention [Problem to be solved by the invention]

[0004] When acquiring the position of a target from an image captured by a camera, an error may occur in the position of the target depending on the camera and the processing method used to process the image, etc. However, the technology of Patent Document 1 does not output information related to the error, and therefore there is a concern that handling the output information may be difficult in applications where precision in the position of the target is required.

[0005] The purpose of the disclosure of this specification is to provide an information processing method that increases the applicability of target information. 。 [Means for solving the problem]

[0006] The information processing method disclosed herein is an information processing method executed by at least one processor (11b, 52b) to output information on a target in a photographing area based on an image of the photographing area photographed by a photographing device (2), the information processing method comprising: Obtaining a probability distribution of pixel errors for each distance range from the image capture device to the image capture area; obtaining an upper limit error of an allowable probability range in the probability distribution for each distance range; The target information includes outputting the target's position in the map coordinate system and the estimated target position error obtained by converting the upper limit error into the range of the map coordinate system.

[0007] According to this information processing method, the target position is output with the estimated position error of the target added to it. The estimated position error is based on the upper limit error of the allowable probability range in the probability distribution of pixel errors. This allows the user of the target information to take into account the target position error due to pixel errors in the image captured by the imaging device, etc., and use the information accordingly. This increases the applicability of the target information.

[0012] Note that the symbols in parentheses included in the claims etc. are intended to exemplify the correspondence with the parts of the embodiments described below, and are not intended to limit the technical scope. [Brief explanation of the drawings]

[0013] [Figure 1] FIG. 1 is a diagram showing a schematic configuration of an infrastructure system and a vehicle. [Figure 2] FIG. 1 is a diagram showing a schematic configuration of a pre-data generation system. [Figure 3] FIG. 1 is a configuration diagram illustrating the functions of a pre-data generation system. [Figure 4] 10 is a flowchart showing an example of pre-processing. [Figure 5] 10 is a flowchart showing an example of a distribution list generation process. [Figure 6] 10 is a flowchart showing an example of an error map calculation process. [Figure 7] A diagram explaining annotations. [Figure 8] FIG. 10 is a diagram illustrating the relationship between bin intervals and pixel error distributions. [Figure 9] FIG. 10 is a diagram illustrating division of bin intervals. [Figure 10] 1 is a graph illustrating the acceptable probability range of pixel error. [Figure 11] FIG. 10 is a diagram illustrating an example of coordinate transformation. [Figure 12] FIG. 2 is a configuration diagram illustrating functions of an information processing device. [Figure 13] 10 is a flowchart showing an example of real-time processing. [Figure 14] FIG. 10 is a diagram illustrating an example of coordinate transformation. [Figure 15] FIG. 10 is a diagram illustrating an example of coordinate transformation. [Figure 16] FIG. 10 is a diagram showing mathematical expressions used in a filter algorithm. [Figure 17] FIG. 3 is a diagram showing mathematical expressions used in the first embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0014] An embodiment will be described with reference to the drawings.

[0015] (First embodiment) An information processing device 10 of the first embodiment shown in Fig. 1 processes images captured by an infrastructure camera 2 installed on the side of a road. The information processing device 10 can provide information based on the results of the image processing to vehicles 90, pedestrians, and the like using the road via a communication device 3. The information processing device 10, together with the infrastructure camera 2 and the communication device 3, may constitute an infrastructure system 1. The infrastructure system 1 may be a roadside communication system.

[0016] The infrastructure camera 2 is a photographing device that is fixedly installed relative to the road and photographs a photographing area including the road. The infrastructure camera 2 includes, for example, an optical system and an imaging element. The optical system includes, for example, one or more lenses, and collects incident light from the photographing area and forms an image on the imaging element. The imaging element is, for example, a CCD sensor or a CMOS sensor, and can generate image data by outputting the light detection results for each of the two-dimensionally arranged pixels, and output the image data to the outside.

[0017] Here, the road photographed by the infrastructure camera 2 may be a road laid within the premises of a facility such as a factory, etc. Also, the road photographed by the infrastructure camera 2 may be a public road.

[0018] The communication device 3 is capable of performing V2X communication, which is wireless communication, with, for example, a communication device 91 of a vehicle 90 traveling on a road. The communication device 3 is, for example, a dedicated short range communications (DSRC) communication device, a cellular V2X (C-V2X) communication device, or the like. The V2X communication may be performed by transmitting and receiving messages based on a predetermined message format. Furthermore, the communication device 3 may communicate with the vehicle 90 by, for example, Bluetooth (registered trademark) communication, Wi-Fi (registered trademark) communication, infrared communication, or the like. Furthermore, the communication device 3 may be provided inside the information processing device 10 as a component of the information processing device 10.

[0019] The information processing device 10 has an interface to the outside and is communicably connected to the infrastructure camera 2 and the communication device 3. As these communication means, for example, a LAN (Local Area Network), a wire harness, an internal bus, a wireless communication circuit, etc. may be adopted.

[0020] The information processing device 10 may be realized mainly by at least one computer 11. The computer 11 may have at least one memory 11a and one processor 11b. The memory 11a may be at least one type of non-transitory tangible storage medium, such as a semiconductor memory, a magnetic medium, or an optical medium, that non-temporarily stores programs and data that can be read by the processor 11b. Furthermore, the memory 11a may be provided with a volatile storage medium, such as a random access memory (RAM), that allows data to be rewritten. The processor 11b includes at least one type of core, such as a central processing unit (CPU), a graphics processing unit (GPU), or a reduced instruction set computer (RISC)-CPU.

[0021] Furthermore, the computer 11 may be a SoC (System on a Chip) in which a memory, a processor, and an interface are integrated into one chip, or may have an SoC as a component of the computer 11.

[0022] The information processing device 10 acquires an image of the shooting area captured by the infrastructure camera 2. The information processing device 10 processes and analyzes the acquired image to detect targets reflected in the shooting area. The information processing device 10 generates information on the detected targets. The information processing device 10 outputs the target information to the communication device 3. In other words, the information processing device 10 outputs the target information to vehicles 90 using the road, etc., via the communication device 3.

[0023] When generating target information, the information processing device 10 refers to the error map 12. The error map 12 is error data that is generated by pre-processing before the information processing device 10 executes real-time processing, and is mapped according to a coordinate system on the image.

[0024] Specifically, the error map 12 is generated using a prior data generation system 50 shown in Fig. 2. The prior data generation system 50 includes, for example, a computer 52, a user interface 51, an image database (hereinafter referred to as image DB) 53, and an annotation database (hereinafter referred to as annotation DB) 54.

[0025] The computer 52 may have at least one memory 52a and one processor 52b. The memory 52a may be at least one type of non-transitory tangible storage medium, such as a semiconductor memory, a magnetic medium, or an optical medium, that non-temporarily stores programs and data that can be read by the processor 52b. The memory 52a may further include a volatile storage medium, such as a random access memory (RAM), that allows data to be rewritten. The processor 52b includes at least one type of core, such as a central processing unit (CPU), a graphics processing unit (GPU), or a reduced instruction set computer (RISC)-CPU.

[0026] The user interface 51 includes a display that displays information to an operator HUM (see also FIG. 3) as a user, and an input device that accepts operation inputs from the operator HUM. The input device may be a keyboard and mouse, or may be a touch panel.

[0027] The image DB 53 is mainly composed of at least one type of non-transient tangible storage medium, such as a semiconductor memory, a magnetic medium, or an optical medium, that non-temporarily stores data that can be read by the processor 52b. The image DB 53 is a database that stores a large number of images of targets to be annotated. The images stored in the image DB 53 may be images of the shooting area previously captured by the infrastructure camera 2.

[0028] The annotation DB 54 is primarily composed of at least one type of non-transient tangible storage medium, such as a semiconductor memory, a magnetic medium, or an optical medium, that non-temporarily stores data that can be read by the processor 52b. The annotation DB 54 is a database for saving annotated images and their annotation results (hereinafter, "annotation data"). The annotation DB 54 stores a large amount of annotation data that has been worked on by the operator HUM.

[0029] As shown in FIG. 3, the advance data generation system 50 includes an annotation tool 61, a distribution list generation tool 62, and an error map calculation tool 63 as functional blocks realized by a processor that executes a program.

[0030] The annotation tool 61 is a tool for preparing annotation data. The annotation tool 61 displays images stored in the image DB 53 on a display. The annotation tool 61 accepts the operation of the operator HUM, which specifies the position where a target object is reflected in the image. The target object position annotated by the operator HUM in this way becomes the correct position when the object recognition result is later evaluated. Hereinafter, data indicating the correct position will be referred to as correct data. The correct data is included in the annotation data. When a bounding box BB is used in the object recognition process, the position may also be specified using a rectangular area RA equivalent to the bounding box BB.

[0031] Next, a distribution list generation tool 62 and an error map calculation tool 63 realize a part of the pre-processing of the information processing method shown in the flowchart of Fig. 4. The processing of steps S1 and S2 shown in Fig. 4 only needs to be executed once in advance for one infrastructure camera 2. In S1, a distribution list 55 is generated. Details of the distribution list generation processing are shown in the flowchart of Fig. 5. In S2, an error map 12 is calculated. Details of the error map calculation processing are shown in the flowchart of Fig. 6.

[0032] The distribution list generation tool 62 is a tool that compares annotation data with object recognition results to calculate pixel errors and generate a pixel error distribution list 55. The distribution list generation process by the distribution list generation tool 62 will be described in detail below.

[0033] The distribution list generation tool 62 acquires annotation data from the annotation DB 54 (S101 in FIG. 5). The distribution list generation tool 62 acquires an image corresponding to the supervised answer data g from the image DB (S102). The image corresponding to the supervised answer data g is the image that was used to create the annotation data. The distribution list generation tool 62 performs object recognition processing on the image (S103). The distribution list generation tool 62 acquires the pixel pi of the target position estimated by the object recognition processing (S104).

[0034] Here, the object recognition process is performed using an object recognition algorithm. The object recognition algorithm may be, for example, an object recognition model, which is a trained model including a neural network. When an image is input to the neural network, the object recognition model recognizes and outputs a target object reflected in the image. The object recognition model outputs a rectangular bounding box BB arranged to surround the target object in the image.

[0035] Meanwhile, the distribution list generation tool 62 acquires pixel pg of the supervised answer data g from the annotation data (S105). The distribution list generation tool 62 calculates a pixel error Δgi between pixel pg and pixel pi (S106). As shown in Fig. 7, the pixel error is a parameter indicating how many pixels pixel pi deviates from pixel pg of the supervised answer data g in the image due to the object recognition process.

[0036] The distribution list generation tool 62 converts pixel pg to position φ(pg) in the map coordinate system using the homography transformation matrix φ and the camera internal parameters (S107). Here, the camera internal parameters include information indicating the optical characteristics of the infrastructure camera 2, such as distortion information. The distortion information may be information that includes in its calculation the installation position and installation angle of the fixedly installed infrastructure camera 2. The camera internal parameters can be used to perform distortion correction of the image. The map coordinate system may be a coordinate system used for maps used in in-vehicle navigation systems, high-precision maps for autonomous driving, etc. The map coordinate system may be a latitude-longitude coordinate system. The map coordinate system may be a coordinate system specified in a message format for V2X communication.

[0037] Furthermore, the distribution list generation tool 62 calculates the distance Lg from the infrastructure camera 2 to φ(pg) (S108).

[0038] The process of comparing the annotation data with the object recognition result to calculate the pixel error and the distance from the infrastructure camera 2 is executed for all annotation data stored in the annotation DB 54. Based on this result, the distribution list generation tool 62 generates pairs of pixel error Δgi and distance Lg for the total number of annotation data (S109).

[0039] Next, the distribution list generation tool 62 bins the distance Lg according to the bin width (S110). Here, the bin width may be set to an equal width for each bin, as shown in FIG.

[0040] On the other hand, the bin width may be set to a different width for each bin so that the bin width increases as the distance from the infrastructure camera 2 increases, as shown in Fig. 9. This configuration provides a width to accommodate the decrease in resolution, since the distance resolution decreases as the distance increases.

[0041] The distribution list generation tool 62 then accumulates the pixel errors Δgi for each bin and generates a distribution PL of the pixel errors Δgi for each bin (S111). The statistically obtained distribution PL can be said to be a probability distribution of the pixel errors Δgi. The distribution list 55 is a list of the distributions PL for each bin, which are divided into distance ranges from the infrastructure camera 2. The distribution list 55 may be stored in the memory 52a, for example, in a state that allows it to be accessed later.

[0042] The error map calculation tool 63 converts pixel errors in the coordinate system on the image for all pixels on the image of the photographed area into distance errors in the map coordinate system and creates a map. Details of the error map calculation process by the error map calculation tool 63 are described below.

[0043] The error map calculation tool 63 calculates the distance L from the infrastructure camera 2 to the pixel to be calculated (S201 in FIG. 5). This calculation may be performed using the above-mentioned camera internal parameters. The error map calculation tool 63 obtains the probability distribution PL corresponding to the distance L from the distribution list 55 (S202).

[0044] 10, the error map calculation tool 63 refers to a preset probability threshold R and acquires a pixel error σL that satisfies formula (1) in FIG. 1 (S203). Formula (1) indicates an allowable probability range, which is a range within which the probability R is allowable for the pixel error probability distribution PL. The pixel error σL indicates the upper limit error in the allowable probability range.

[0045] The error map calculation tool 63 acquires pixel k that is σL away from pixel p in the image of the infrastructure camera 2 (S204). As shown in Fig. 11, the error map calculation tool 63 converts pixel p and pixel k to positions φ(p) and φ(k) in the map coordinate system using the homography transformation matrix φ and the camera internal parameters (S205). The error map calculation tool 63 calculates the distance dpk between φ(p) and φ(k) (S206).

[0046] Here, there are multiple pixels k for pixel p. Therefore, it is advisable to first calculate the distance dpki for all pixels k, and then select one pixel k to be used from the multiple pixels k using equation (2) which shows the evaluation function in FIG. 17. The evaluation function of equation (2) is a function that selects, from the multiple pixels k, the pixel k that has the maximum distance dpk. Note that pixel p may be referred to as a reference pixel, and pixel k may be referred to as a neighboring pixel.

[0047] As a result, a distance dpk is determined for one selected pixel k. The distance dpk represents a position error in the map coordinate system with probability R. In other words, the probability that the target position φ(p) has an error of distance dpk is R.

[0048] This process of calculating the distance dpk is performed for all pixels p that make up the image. Once the distance dpk has been calculated for all pixels p, the error map calculation tool 63 stores the distance dpk for each pixel p in the error map 12 as a position error of the pixel p (S207). In this way, the error map 12 is generated according to the installation status of the infrastructure camera 2.

[0049] The error map 12 generated in this manner may be permanently stored in, for example, the memory 11a in the information processing device 10. Alternatively, the information processing device 10 may include a dedicated storage medium for storing the error map 12. The information processing device 10 processes images from the infrastructure camera 2 in real time and can sequentially provide information on targets.

[0050] As shown in Fig. 12, the information processing device 10 includes an object recognition unit 21, a coordinate conversion unit 22, an error region calculation unit 23, a filter unit 24, and a target integration unit 25 as functional blocks realized by a processor that executes a program. These processing units 21 to 25 realize a part of the real-time processing of the information processing method shown in the flowchart of Fig. 13. The processing of steps S301 to S308 shown in Fig. 4 is executed, for example, every time a new image is provided from the infrastructure camera 2.

[0051] For example, when a video signal output for each image frame from the infrastructure camera 2 is captured using RTSP (Real Time Streaming Protocol) (S301 in FIG. 13), the object recognition unit 21 executes object recognition processing (S302). Specifically, the object recognition unit 21 applies an object recognition algorithm to the input image frame and outputs the position or area of ​​the target.

[0052] The object recognition algorithm should essentially be the same as the algorithm used in the pre-processing, because the error map 12 used later is a map that depends on the performance of the object recognition algorithm.

[0053] The coordinate transformation unit 22 transforms each position in the target area represented in the coordinate system on the image into a position in the map coordinate system using the homography transformation matrix φ and the camera's internal parameters. Specifically, the coordinate transformation unit 22 first acquires pixel p at the target position estimated by the object recognition process (S303). The coordinate transformation unit 22 calculates position φ(p) in the map coordinate system corresponding to pixel p using the homography transformation matrix φ and the camera's internal parameters (S304).

[0054] The error region calculation unit 23 obtains information on the position error assigned to the pixel to be coordinate converted from the error map 12, and generates a position error region EA for the position in the map coordinate system corresponding to the pixel. Specifically, the error region calculation unit 23 obtains a position error dpk corresponding to pixel p from the error map 12 (S305). The error region calculation unit 23 applies the position error dpk as an estimated position error estimated for the target, and calculates an area from the position φ(p) that is equal to or smaller than the position error dpk as the position error region EA (S306).

[0055] Here, in FIG. 14, targets are handled using a two-dimensional coordinate system. The left part of FIG. 14 shows targets on an image. The right part of FIG. 14 shows targets on a map coordinate system. Pixel p may be a pixel corresponding to the center point of the contact portion where the target's bounding box BB is in contact with the road. In the map coordinate system, the position φ(p) may be represented by a point corresponding to the center point of the target. The position error area EA may be represented by a circular area surrounded by a circle centered at the position φ(p).

[0056] Furthermore, in FIG. 15, targets are handled using a three-dimensional coordinate system. The left part of FIG. 15 shows targets on an image. The right part of FIG. 15 shows targets on a map coordinate system. In this case, the bounding box BB of the target may be represented by a three-dimensional figure so as to include information on the three-dimensional size of the target. The three-dimensional figure may be, for example, a rectangular parallelepiped. When the target is represented by a rectangular parallelepiped, pixel p may be a pixel corresponding to four points of the part of the rectangular parallelepiped that is in contact with the road.

[0057] The target in the map coordinate system may also be represented so as to include size information. For example, when the map is represented in a two-dimensional coordinate system, the target position may be represented by a rectangular area surrounded by four positions φ(p) obtained by coordinate transformation of pixel p corresponding to the above-mentioned four points. The position error area EA may be represented by a rectangular area obtained by further enlarging the rectangular area indicating the target position by the position error dpk.

[0058] 16, the filter unit 24 uses the position error calculated by the error region calculation unit 23 as a target observation error parameter of the filtering algorithm (S307). In detail, the filter unit 24 acquires the position error dpk of the target and uses the position error dpk as the observation noise R when updating the Kalman filter. The filtering algorithm is, for example, a linear Kalman filter or an extended Kalman filter (EKF).

[0059] The target integrating unit 25 integrates targets using the positions filtered by the filter unit 24 (S308). For example, if the object recognition unit 21 recognizes one target as two bounding boxes BB, it determines whether to integrate the two bounding boxes BB based on predictions by the Kalman filter. When providing target information, the target integration process determines the allocation of an ID to the target.

[0060] In addition to being used directly to calculate the position error area EA, the position error dpk can also be used to calculate the position error area EA on map coordinates using the covariance of the Kalman filter after target integration using algorithms such as the Hungarian Algorithm, PDAF, and JPDAF.

[0061] In this way, the target information is processed and output. Here, the information processing device 10 outputs the position of the target in the map coordinate system and the estimated position error of the target in the map coordinate system. The estimated position error may be output as the above-mentioned position error area EA. Also, the estimated position error may be output as the value of the above-mentioned position error dpk.

[0062] Next, a vehicle 90 that receives target information will be described. As shown in Fig. 1 , the vehicle 90 includes a communication device 91 and a display system 92. The communication device 91 may have the same configuration as the communication device 3 as long as it is capable of V2X communication with the communication device 3 provided in the infrastructure system 1. The communication device 91 provides the target information received from the infrastructure system 1 to the display system 92.

[0063] The display system 92 includes a computer 93 and a display device 94. The computer 93 may include at least one memory 93a and one processor 93b. The memory 93a may be at least one type of non-transitory tangible storage medium, such as a semiconductor memory, a magnetic medium, or an optical medium, that non-temporarily stores programs and data readable by the processor 93b. The memory 93a may further include a volatile storage medium, such as a random access memory (RAM), that allows data to be rewritten. The processor 93b includes at least one type of core, such as a central processing unit (CPU), a graphics processing unit (GPU), or a reduced instruction set computer (RISC)-CPU.

[0064] The computer 93 executes processing related to the display content and outputs a video signal to the display device 94 so as to display the display content. The display device 94 is, for example, at least one type of device selected from the group consisting of a graphic meter, a combination meter, a navigation unit, a CID (Center Information Display), and a HUD (Head-Up Display). The display system 92 may be provided with a plurality of display devices 94. The display device 94 can display one or both of a real image and a virtual image to the driver of the vehicle 90 or to passengers including the driver.

[0065] The computer 93 generates display content related to the target information provided by the infrastructure system 1 and provides the display content to the driver or passengers via the display device 94. The display content may include a target information image that integrally combines the target position and the target position error area EA, as shown in the right parts of FIGS. 14 and 15 . A plurality of target information images may be displayed simultaneously so that the relative positional relationships between a plurality of targets are visually represented. The display content may further include a vehicle image in which the vehicle 90 is displayed from a bird's-eye view so that the relative positional relationship between the target position and the vehicle 90 is visually represented. The relative positional relationship includes at least one of a directional relationship and a distance relationship.

[0066] The first embodiment described above is summarized below. According to the first embodiment, the target position is output after adding the estimated position error of the target. The estimated position error is based on the upper limit error of the allowable probability range in the probability distribution of pixel errors. This allows the target information user to take into account and use the target position error based on pixel errors, etc. in the image captured by the infrastructure camera 2 as a capturing device. This increases the applicability of the target information.

[0067] The pixel error probability distribution is the distribution of errors on the image pixels between the target position recognized from the image by the target recognition algorithm and the correct target position input by annotation. Because the pixel error probability distribution is generated based on the target algorithm, it is possible to reflect not only errors caused by the infrastructure camera 2 but also errors caused by the algorithm.

[0068] Furthermore, among multiple neighboring pixels k that are far from the origin pixel p by the upper limit error, the neighboring pixel k with the largest distance dpk is selected, and the distance of the selected neighboring pixel k is taken as the estimated position error. Since the error is estimated from the neighboring pixel k with the greatest influence, the user of the target information can consider the error with greater confidence.

[0069] Furthermore, the target position is output with the estimated position error of the target added to it. The estimated position error is obtained by referring to error data that stores the estimated position error in the map coordinate system associated with the pixel. This allows the target information user to take into account the target position error based on pixel error, etc. in the image captured by the infrastructure camera 2. This increases the applicability of the target information.

[0070] The estimated position error is output as a position error area EA, which indicates the error of the target's estimated position in the map coordinate system. By showing the area where the target may exist, the target information can be easily utilized.

[0071] Furthermore, the target position is estimated through a Kalman filter using the estimated position error as an update parameter, which significantly improves the accuracy of target position estimation.

[0072] Furthermore, a target information image that integrally combines the target position and the position error area EA is presented to the occupant of the vehicle 90. Therefore, the occupant can recognize the target information including the target position error, and can be urged to take appropriate action taking the error into consideration.

[0073] (Other embodiments) Although one embodiment has been described above, the present disclosure should not be construed as being limited to this embodiment, and can be applied to various embodiments within the scope of the gist of the present disclosure.

[0074] For example, the image capturing device may be an in-vehicle camera that is fixedly installed on the vehicle and captures images of the periphery of the vehicle, instead of the infrastructure camera 2. The image capturing device may be a sensor such as LiDAR (Light Detection and Ranging / Laser Imaging Detection and Ranging) or imaging radar that can generate images from detection results.

[0075] The information processing device 10 may be mounted on a vehicle and output target information to a display system, an automatic driving system, etc. of the vehicle. The information processing device 10 may be a component of a display system, an automatic driving system, etc. of the vehicle, and may output target information to a display device, an automatic driving system, etc. within the system.

[0076] When the information about the target object is used for autonomous driving or driving assistance, the estimated position error may be used as a safety margin to avoid collision with the target object.

[0077] In the error map 12, the position error dpk associated with a pixel may be anisotropic, for example, the position error dpk may be expressed as a function of direction.

[0078] The information processing device 10 may not have the filter unit 24 and the target integrating unit 25. In other words, the position of the target and the position error area EA may be output at the time when the error area calculation unit 23 calculates the position error area EA.

[0079] The controller and methods described herein may be implemented by a special-purpose computer comprising a processor programmed to perform one or more functions embodied in a computer program. Alternatively, the apparatus and methods described herein may be implemented by special-purpose hardware logic circuitry. Alternatively, the apparatus and methods described herein may be implemented by one or more special-purpose computers comprising a processor executing a computer program in combination with one or more hardware logic circuits. Furthermore, the computer program may be stored as instructions executed by a computer on a computer-readable non-transitory storage medium. [Explanation of symbols]

[0080] 2: infrastructure camera (photographing device), 10: information processing device, 11b, 52b: processor, 12: error map (error data), 90: vehicle, 92: display system, 93: computer, 94: display device, EA: position error area

Claims

1. An information processing method executed by at least one processor (11b, 52b) to output information on a target in a photographed area based on an image of the photographed area photographed by a photographing device (2), comprising: acquiring a probability distribution of pixel errors for each distance range from the image capture device to the image capture area in the image; obtaining an upper limit error of an allowable probability range in the probability distribution for each of the distance ranges; and outputting, as the information about the target, the position of the target in a map coordinate system and an estimated position error of the target obtained by converting the upper limit error into a range of the map coordinate system.

2. 2. The information processing method according to claim 1, wherein the probability distribution of pixel errors is a distribution of errors on pixels of the image between the position of the target recognized from the image by a target recognition algorithm and the correct position of the target input by annotation.

3. Define the distance between a source pixel in the image, a neighboring pixel that is the upper limit error away from the source pixel, and a position where the source pixel is transformed into the map coordinate system and a position where the neighboring pixel is transformed into the map coordinate system as follows:

3. The information processing method according to claim 1, further comprising: selecting, from among a plurality of neighboring pixels that are farther from the origin pixel than the upper limit error, the neighboring pixel with the longest distance; and setting the distance of the selected neighboring pixel as the estimated position error.

Images