Image Processing Method

Abstract

To appropriately conceal information even when it is difficult to recognize a license plate of another vehicle included in a camera image.SOLUTION: A camera image is processed based on the camera image in front of or behind a vehicle and point cloud data obtained from a lidar. Firstly, an image region that may include an image of another vehicle is specified in the camera image. Subsequently, when a size of a three-dimensional object present in front of or behind the vehicle estimated based on the point group data is equal to or larger than a vehicle equivalent size, an image candidate of the other vehicle is extracted from the image region specified in the camera image in a direction in which the three-dimensional object is present. Subsequently, a predetermined region in the image candidate is masked. The predetermined region includes a region below a center of the image candidate, and has a certain spread in a horizontal direction from the center. Then, the image candidate in which the predetermined region is masked is superimposed on the image region of the camera image from which the image candidate is extracted.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present disclosure relates to a method for processing images acquired by a camera mounted on a vehicle (hereinafter also referred to as "camera images"). [Background technology]

[0002] Japanese Patent Application Laid-Open Publication No. 2019-200728 discloses a method for, when another vehicle is recognized in a camera image of a vehicle, further recognizing the license plate of the other vehicle and applying mosaic processing to the image area where the license plate is recognized. By using the mosaic processing, the information on the license plate of the other vehicle can be concealed, so that the personal information of the owner of the other vehicle can be protected. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2019-200728 A [Patent Document 2] JP 2006-178825 A [Patent Document 3] JP 2017-103748 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, if the surface of the license plate is dirty due to mud splashes or other reasons, the license plate may not be recognized from the camera image. In that case, there is a risk that the mosaic processing will not be performed appropriately.

[0005] One object of the present invention is to provide a technique that can appropriately conceal information about the license plate of another vehicle included in a camera image even when the license plate information is difficult to recognize. [Means for solving the problem]

[0006] The present disclosure relates to a method for processing camera images of the front or rear of a vehicle, the method having the following features: The method comprises: acquiring the front or rear camera image; A step of processing data of reflected light of a laser pulse output from the vehicle and reflected at reflection points around the vehicle to obtain point cloud data; processing the front or rear camera image based on the front or rear camera image and the point cloud data; Equipped with. The step of processing the front or rear camera image includes: identifying an image area in the front or rear camera image that may include an image of another vehicle; extracting image candidates of the other vehicle from an image area identified in a camera image in a direction in which the three-dimensional object exists, when a size of the three-dimensional object existing in front of or behind the vehicle estimated based on the point cloud data is equal to or larger than a size equivalent to the vehicle; a step of masking a predetermined area in the candidate image, the predetermined area including an area below a center of the candidate image and having a certain extent in both left and right directions from the center; A step of superimposing the image candidate with the predetermined region masked on an image region of the camera image from which the image candidate was extracted; Includes. Effect of the Invention

[0007] According to the present disclosure, an image area that may include an image of another vehicle is specified in a camera image of the front or rear. In addition, when the size of a three-dimensional object existing in front or rear of a vehicle estimated based on point cloud data is equal to or larger than the size of the vehicle, a candidate image of the other vehicle is extracted from the image area specified as being likely to include an image of the other vehicle in the camera image in the direction in which the three-dimensional object exists. Then, a predetermined area in this candidate image is masked. This predetermined area includes an area below the center of the candidate image of the other vehicle, and has a certain extent in the left and right directions from this center.

[0008] According to the present disclosure, the image candidate with a masked predetermined area is further superimposed on the image area of ​​the camera image from which the image candidate is extracted. By processing the series of camera images, a predetermined area of ​​the image area identified as possibly including an image of another vehicle is masked. This predetermined area corresponds to the position of the vehicle body to which the license plate is attached. Therefore, according to the present disclosure, even if it is difficult to recognize the license plate of another vehicle included in the camera image, the information can be appropriately concealed. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating an example of the configuration of a system for implementing an image processing method according to an embodiment of the present disclosure. [Diagram 2] FIG. 2 is a diagram illustrating an example of a camera image of a front view of a vehicle. [Diagram 3] FIG. 2 is a diagram illustrating an example of point cloud data of a front area of ​​a vehicle. [Figure 4] 2 is a flowchart showing an example of processing by the data processing device (processor) shown in FIG. 1. [Diagram 5] FIG. 5 is a diagram illustrating the process of step S12 shown in FIG. [Figure 6] FIG. 5 is a diagram illustrating the process of step S13 shown in FIG. [Figure 7] FIG. 13 is a diagram showing the size of the point cloud group extracted in the processing of step S13. [Figure 8] FIG. 13 is a diagram illustrating an example of a mask process for a predetermined area. [Figure 9] FIG. 5 is a diagram showing an example of an image synthesized by the process of step S17 shown in FIG. [Figure 10] 1A and 1B are diagrams illustrating an example of concealing the license plate of a following vehicle included in a camera image behind a vehicle. [Figure 11] FIG. 11 is a diagram illustrating another example of a camera image in front of the vehicle. [Figure 12]1A and 1B are diagrams illustrating an example of concealing the license plate of another preceding vehicle included in a camera image in front of the vehicle. [Figure 13] FIG. 13 is a diagram showing another example of a camera image in front of the vehicle. [Figure 14] 1A and 1B are diagrams illustrating an example of concealing the license plate of a vehicle traveling ahead that is included in a camera image in front of the vehicle. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an image processing method according to an embodiment of the present disclosure will be described with reference to the drawings. Note that in each drawing, the same or corresponding parts are denoted by the same reference numerals, and the description thereof will be simplified or omitted.

[0011] 1. System configuration example FIG. 1 is a diagram for explaining a configuration example of a system for implementing an image processing method according to an embodiment of the present disclosure. The system 100 shown in FIG. 1 is a system mounted on a vehicle and communicates with an external server 200. Hereinafter, the vehicle on which the system 100 is mounted is also referred to as a "host vehicle". The system 100 and the external server 200 constitute a data storage system that stores, for example, camera images acquired by a camera mounted on the host vehicle together with data related to the running environment of the host vehicle at the time the camera images were acquired. The stored data is used, for example, for analyzing traffic flow.

[0012] In the example shown in FIG. 1, the system 100 includes a camera 10, a LiDAR (Laser Imaging Detection And Ranging) 20, an internal sensor 30, a GNSS (Global Navigation Satellite System) device 40, a map database 50, a communication device 60, and a data processing device 70.

[0013] The camera 10 captures at least the front and rear of the vehicle to obtain a camera image CAM. The camera image CAM is transmitted to the data processing device 70. The camera 10 may process the camera image CAM to recognize objects around the vehicle. In this case, object recognition data is transmitted from the camera 10 to the data processing device 70 separately from the camera image CAM. Examples of the object recognition data include data indicating the attributes of the object, data indicating the distance from the vehicle to the object, data indicating the direction of the object relative to the vehicle, and data indicating the position of the object.

[0014] The LIDAR 20 is a flash or rotary type device that outputs a laser pulse around the vehicle. When the laser pulse is reflected at a reflection point around the vehicle, the reflected light of the laser pulse returns to the LIDAR 20. The LIDAR 20 obtains data of the reflection points by detecting the reflected light. The data of the reflection points includes, for example, data of the output angle of the laser pulse and data of the time of flight (TOF: Time Of Flight) of the laser pulse at this output angle. When the data of the reflection points is processed, point cloud data LID is obtained. Each piece of data constituting the point cloud data LID includes, for example, data of a 3D position in a sensor coordinate system fixed to the LIDAR 20 (i.e., X, Y, and Z position data). The point cloud data LID is transmitted to the data processing device 70.

[0015] The internal sensor 30 detects the running state of the host vehicle. Examples of the internal sensor 30 include a wheel speed sensor, an acceleration sensor, and a yaw rate sensor. The wheel speed sensor detects the running speed of the host vehicle. The acceleration sensor detects the acceleration of the host vehicle. The yaw rate sensor detects the yaw rate around the vertical axis of the center of gravity of the host vehicle. These data are collectively referred to as "sensor data SEN." The sensor data SEN is transmitted to the data processing device 70. The sensor data SEN is an example of the above-mentioned "data related to the running environment."

[0016] The GNSS device 40 receives signals from three or more artificial satellites. The GNSS device 40 generates position and attitude data POS of the vehicle based on the received signals. The position and attitude data POS is transmitted to the data processing device 70. The position and attitude data POS is also an example of the above-mentioned "data related to the driving environment."

[0017] Map data is stored in the map database 50. Examples of the map data include road position data, road shape data (e.g., types of curves and straight lines), and intersection and structure position data. The map database 50 is formed in a predetermined storage device (e.g., a hard disk, a flash memory) mounted on the vehicle. The map database 50 may also be formed in an external computer (e.g., an external server 200) capable of communicating with the vehicle.

[0018] The communication device 60 performs wireless communication with a base station (not shown) on the network. Examples of communication standards for this wireless communication include mobile communication standards such as 4G, LTE, and 5G. The connection destination of the communication device 60 includes at least an external server 200. In communication with the external server 200, the communication device 60 transmits, for example, the ID data of the vehicle itself, the camera image CAM, data on the time when the camera image CAM was acquired, and the above-mentioned "data related to the driving environment." When image processing, which will be described later, is performed, the camera image after image processing is transmitted instead of the camera image CAM. When the "data related to the driving environment" is transmitted, these data are data acquired at the same time as the time when the camera image CAM was acquired.

[0019] The data processing device 70 is composed of a microcomputer having at least one processor 71 and at least one memory 72. The processor 71 includes a CPU (Central Processing Unit). The memory 72 is a volatile memory such as a DDR memory, and expands programs used by the processor 71 and temporarily stores various data. The various data include a camera image CAM, point cloud data LID, sensor data SEN, and position and orientation data POS.

[0020] The processor 71 executes various types of processing according to the programs stored in the memory 72. The various types of processing include alignment (calibration) processing of the camera 10 and the lidar 20, processing to recognize objects included in the camera image CAM, processing of the camera image CAM (image processing), etc. Publicly known techniques are applied to the alignment processing and object recognition processing. The processing of the camera image CAM is processing related to the image processing method according to the embodiment, and will be described later.

[0021] 2. Features of the embodiment 2-1. Problems with object recognition processing using camera image CAM With reference to FIG. 2, problems with object recognition processing using a camera image CAM will be described. FIG. 2 is a diagram that shows an example of a camera image CAM in front of the host vehicle. FIG. 2 shows a rear part of a leading vehicle PV1. The leading vehicle PV1 is an example of another vehicle that travels in the same lane as the host vehicle in the same direction as the host vehicle in the same lane as the lane L1 in which the host vehicle travels. The lane L1 is a lane that is defined between a curbstone CU1 shown in FIG. 2 and a center line CL drawn on the road. The lane L1 may be defined by the center line CL and a white line drawn on the road similar to the center line CL. Note that FIG. 2 also shows a lane L2 that is defined between the center line CL and a curbstone CU2. Like the lane L1, the lane L2 may be defined by the center line CL and a white line.

[0022] The preceding vehicle PV1, the curbs CU1 and CU2, and the center line CL are recognized by object recognition processing by the camera 10 or the processor 71. In the object recognition processing, it is also possible to recognize the license plate NP1 of the preceding vehicle PV1. However, if the surface of the license plate NP1 is dirty due to mud splashes or the like, there is a possibility that the license plate NP1 included in the camera image CAM will not be recognized.

[0023] In addition, while the center line CL is painted in a conspicuous color (e.g., white or yellow), the body color of the leading vehicle PV1 varies. Therefore, depending on the scenery around the leading vehicle PV1, the leading vehicle PV1 included in the camera image CAM may not be recognized. If this happens, the license plate NP1 may be mistaken for an object other than a "license plate."

[0024] 2-2. Estimating the presence of other vehicles using point cloud data LID In a system that transmits the camera image CAM to the external server 200, it is important to conceal personal information contained in the camera image CAM, i.e., license plate information. In this embodiment, taking into consideration the case where the license plate NP1 is not recognized or the license plate NP1 is mistaken for an object other than a license plate, the object recognition process using the camera image CAM goes as far as identifying an "image area that may include an image of another vehicle." The "image area that may include an image of another vehicle" is, for example, the road around the vehicle, and is an area that includes the lanes L1 and L2 shown in FIG. 2. The lanes L1 and L2 are identified by recognizing the center line CL and the curbs CU1 and CU2.

[0025] In the embodiment, the presence of other vehicles around the vehicle is further estimated using the point cloud data LID. Fig. 3 is a diagram showing an example of the point cloud data LID in front of the vehicle. The point cloud data LID is generated based on the data of the reflection points acquired at the same time as the camera image CAM in Fig. 2 is acquired. Fig. 3 shows the point cloud data LID corresponding to the rear part of the preceding vehicle PV1 and the point cloud data LID corresponding to the curbs CU1 and CU2. For convenience of explanation, the point cloud data based on the data of the reflection points on the road is omitted.

[0026] The point cloud data LID includes Z position data indicating the distance from the LIDAR 20 to the object. Therefore, for example, point cloud data having similar values ​​of Z position data and similar values ​​of X and Y position data are grouped (clustering process). Then, a point cloud group corresponding to the rear part of the preceding vehicle PV1 is specified. Also, for example, point cloud data having continuously changing values ​​of Z position data and similar values ​​of X and Y position data are grouped. Then, point cloud groups corresponding to the curbs CU1 and CU2 are specified. Incidentally, such grouping process of the point cloud data LID is a known technique, and therefore further explanation is omitted.

[0027] Once the point cloud group is identified, it is determined whether each size of the point cloud group is equal to or larger than the vehicle size. Here, the vehicle size is specified by the width W and height H, and is set to a size that excludes fallen objects smaller than the size of a general vehicle, such as cardboard boxes on a road. The vehicle size may be set to the same size in front of and behind the vehicle, or different sizes may be set.

[0028] In the embodiment, a point cloud group whose size is equal to or larger than the size corresponding to a vehicle is estimated to be that of another vehicle. Then, an image that corresponds to the position where the point cloud group of the other vehicle is identified and has the same size as the size of the point cloud group is extracted from the "image area that may include an image of the other vehicle". The extracted image is not an image recognized as another vehicle in the object recognition process using the camera image CAM, but an image estimated to be another vehicle in the presence estimation process using the point cloud data LID, and is therefore hereinafter also referred to as a "candidate image of another vehicle".

[0029] In this manner, in the embodiment, by combining the object recognition process using the camera image CAM and the presence estimation process using the point cloud data LID, image candidates of other vehicles existing around the vehicle are extracted from the camera image CAM. Therefore, even if the license plate of another vehicle is not recognized by the object recognition process using the camera image CAM, or if the license plate is mistaken for an object other than a "license plate," it is possible to extract image candidates of other vehicles that are likely to include an image of the license plate from the camera image CAM.

[0030] An example of concealing the license plate of another vehicle using candidate images of the other vehicle will be described in the description of FIG.

[0031] 2-2. Example of image processing by a data processing device (processor) Fig. 4 is a flowchart showing an example of processing by the data processing device 70 (processor 71) shown in Fig. 1. Note that the image processing method according to the embodiment of the present disclosure is realized by executing the processing of steps S11 to S17 in Fig. 4. The routine shown in Fig. 4 is repeatedly executed, for example, while the ignition switch of the host vehicle is turned on.

[0032] 4, first, the point cloud data LID and the camera image CAM are acquired (step S11). As already described, the acquired point cloud data LID is generated based on the data of the reflection points acquired at the same time as the camera image CAM is acquired.

[0033] Following the processing of step S11, "an image area that may include an image of another vehicle" is identified (step S12). FIG. 5 is a diagram for explaining the processing of step S12. FIG. 5 illustrates the same camera image CAM as FIG. 2. However, in the processing of step S12, the recognition result of the other vehicle by the object recognition processing using the camera image CAM is not used, and an image area IA that may include an image of the other vehicle is identified based on the recognition result of the center line CL and the curbs CU1 and CU2. The image area IA shown in FIG. 5 corresponds to the area surrounded by the curbs CU1 and CU2.

[0034] Following the processing of step S12, a grouping process of the point cloud data LID is performed (step S13). FIG. 6 is a diagram for explaining the processing of step S13. FIG. 6 depicts the same point cloud data LID as FIG. 3. When the grouping process of the point cloud data LID is performed, point cloud groups GP (GP1 to GP3) are identified. The identified point cloud groups GP are extracted from the point cloud data LID. At the time when this grouping process is completed, it is unclear what object's reflection point data the identified point cloud group GP is derived from.

[0035] Following the process of step S13, the point cloud group GP identified in step S12 is evaluated (step S14). In the process of step S14, for example, the size (width W and height H) of the point cloud group GP is calculated for each identified point cloud group GP based on the values ​​of the X and Y position data constituting the point cloud group GP. Figure 7 is a diagram showing the size (width W and height H) of the point cloud group GP1 extracted in the process of step S13.

[0036] Following the process of step S14, it is determined whether or not another vehicle is present around the host vehicle (step S15). In the process of step S15, the size of the point cloud group GP calculated in the process of step S14 is compared with the vehicle-equivalent size. This comparison is performed for all point cloud groups GP whose sizes have been calculated in the process of step S14. Then, if any point cloud group GP whose size is equal to or larger than the vehicle-equivalent size is included, it is determined that another vehicle is present.

[0037] If it is determined that another vehicle is present, the process of step S16 is performed. If it is determined that another vehicle is not present, the process of step S18 is performed. In the process of step S16, first, an image that corresponds to the position of the point cloud data LID from which the point cloud group GP whose size is determined to be equal to or larger than the vehicle size is extracted and has the same size as the size of this point cloud group GP is extracted from the camera image CAM acquired in the process of step S11. This extracted image corresponds to the above-mentioned "image candidate of another vehicle."

[0038] In the process of step S16, a predetermined area of ​​the "image candidate of another vehicle" extracted from the camera image CAM is subsequently masked (for example, mosaic-processed, painted over, etc.). FIG. 8 is a diagram showing an example of the mask process of a predetermined area. In the example shown in FIG. 8, first, the center CP of the image candidate IM_CD of the other vehicle is specified. Then, a mask MS is applied to an area including an area below this center CP and having a certain spread in the left and right directions from this center CP. As the width in the left and right direction, for example, a value a obtained by multiplying a value b (=W / 2) obtained by equally dividing the image candidate IM_CD in the height H direction passing through the center CP by a coefficient c (0.4≦c≦0.6) can be considered. Hereinafter, the image candidate IM_CD to which the mask MS has been applied is also referred to as a "masked image IM_MS".

[0039] Following the processing of step S16, image synthesis is performed (step S17). In the processing of step S17, a masked image IM_MS is superimposed on the area of ​​the camera image CAM from which the image candidate IM_CD was extracted in the processing of step S16. FIG. 9 is a diagram showing an example of an image synthesized by the processing of step S17. The synthesized image CAM_C shown in FIG. 9 is generated from the camera image CAM acquired in the processing of step S11 and the masked image IM_MS generated by the processing of step S16.

[0040] In the process of step S18, data is transmitted. The data is transmitted to the external server 200, specifically by outputting various data to the communication device 60. The breakdown of the various data varies depending on the determination result of step S15. Specifically, if the determination result of step S15 is negative, data including the camera image CAM acquired in the process of step S11 is output. On the other hand, if the determination result of step S15 is positive, data including the composite image CAM_C generated in the process of step S17 is output.

[0041] 3. Another example of image processing 2 to 9, an example of image processing for concealing the license plate NP1 of the preceding vehicle PV1 included in the camera image CAM in front of the host vehicle has been described. In the following, an example of applying the image processing method according to the embodiment to a vehicle other than the preceding vehicle PV1 will be described.

[0042] 10 is a diagram illustrating an example of concealing the license plate of a following vehicle included in a camera image CAM of the rear of the host vehicle. The following vehicle is an example of another vehicle traveling in the same lane as the host vehicle and in the same direction as the host vehicle.

[0043] In the case of a following vehicle, the camera image CAM includes the front part of the vehicle. In this case, as shown in FIG. 10, the image candidate IM_CD may include a face image of the driver (and passengers). The face image information, like the license plate information, can be said to be personal information included in the camera image CAM. Therefore, in the example shown in FIG. 10, a mask MS is applied to a wider area than the area described in FIG. 8.

[0044] 10, a mask MS is applied to an area including the areas above and below the center CP of the image candidate IM_CD and having a certain extent in the left and right directions from the center CP. The height (width in the height H direction) of the area above the center CP can be, for example, a value d obtained by multiplying a coefficient f (0.4≦f≦0.6) by a value e (=H / 2) obtained by equally dividing the image candidate IM_CD in the width W direction passing through the center CP.

[0045] 11 and 12 are diagrams illustrating an example of concealing the license plate of another preceding vehicle included in a camera image CAM in front of the host vehicle. The preceding vehicle PV2 shown in Fig. 11 is an example of another vehicle traveling in the same direction as the host vehicle in a lane L2 adjacent to the lane L1 in which the host vehicle is traveling.

[0046] Since the preceding vehicle PV2 is traveling in lane L2, the camera image CAM includes images of the rear and side of the preceding vehicle PV2. However, for example, in the grouping process of the point cloud data LID, by tightening the threshold value for determining that the Z position data is close, the size extracted as the point cloud group GP becomes smaller, and the size of the image candidate IM_CD extracted from the "image area that may include images of other vehicles" also becomes smaller. In this way, as shown in FIG. 12, the image candidate IM_CD is obtained from which part of the image of the side has been removed.

[0047] In the example shown in Fig. 12, a mask MS is applied based on the same method as that described in Fig. 8. This conceals the information on the license plate NP2 of the preceding vehicle PV2.

[0048] 10 to 12, it can be understood that the image processing method according to the embodiment is also applied to an oncoming vehicle included in the camera image CAM in front of the vehicle. That is, since the oncoming vehicle is an example of another vehicle traveling in a lane adjacent to the lane in which the vehicle is traveling, in a direction approaching the vehicle, the camera image CAM includes images of the front and side of the oncoming vehicle. Therefore, by combining the extraction method of the image candidate IM_CD described in FIGS. 11 and 12 with the mask MS described in FIG. 10, the information on the license plate of the oncoming vehicle and the face image of the driver are concealed.

[0049] 13 and 14 are diagrams for explaining an example of concealing the license plate of the vehicle ahead of the vehicle included in the camera image CAM in front of the vehicle. The vehicle ahead of the vehicle PV3 shown in Fig. 13 is an example of another vehicle traveling in the same direction as the vehicle ahead of the vehicle PV1 in the lane L1 in which the vehicle is traveling. If the vehicle ahead of the vehicle PV1 and the vehicle ahead of the vehicle PV3 are in different lateral (vehicle width) positions or if the lane L1 is curved, the camera image CAM will include an image of the rear part of the vehicle ahead of the vehicle PV3.

[0050] Therefore, in the example shown in Fig. 14, a mask MS is applied based on the same method as that described in Fig. 8. This conceals the information on the license plate NP3 of the vehicle before the preceding vehicle PV3.

[0051] 4. Effects of the embodiment According to the image processing method of the embodiment described above, even when it is difficult to recognize the license plate of another vehicle included in the camera image CAM, it is possible to appropriately conceal the information.

[0052] 5. Other embodiments A part of the processing by the data processing device 70 (processor 71) may be executed by the external server 200. For example, masking of a predetermined area of ​​the image candidate IM_CD may be executed by the processor of the external server 200. In this case, the camera image CAM and information on the position of the image candidate IM_CD of another vehicle extracted from this camera image CAM are transmitted to the external server 200. [Explanation of symbols]

[0053] 10. Camera 20 Rider 70 Data Processing Device 71 Processor 72 Memory GP1, GP2, GP3 point cloud groups NP1,NP2 license plate PV1,PV2 Leading vehicle PV3 Preceding vehicle IA Image area that may contain images of other vehicles IM_CD Candidate images of other vehicles IM_MS Masked image CAM Camera image CAM_C composite image LID point cloud data

Claims

[Claim 1] 1. A method for processing front or rear camera images of a vehicle, comprising the steps of: acquiring the front or rear camera image; A step of processing data of reflected light of a laser pulse output from the vehicle and reflected at reflection points around the vehicle to obtain point cloud data; processing the front or rear camera image based on the front or rear camera image and the point cloud data; Equipped with The step of processing the front or rear camera image includes: identifying an image area in the front or rear camera image that may include an image of another vehicle; a step of extracting image candidates of the other vehicle from an image area identified as possibly including an image of the other vehicle in a camera image in a direction in which the three-dimensional object exists, when the size of the three-dimensional object existing in front of or behind the vehicle estimated based on the point cloud data is equal to or larger than the size of the vehicle, wherein an extraction position of the image candidate corresponds to the position of the three-dimensional object and an extraction size of the image candidate is the same as the size of the three-dimensional object; A step of masking a predetermined area in the extracted image candidate, the predetermined area including an area below a center of the extracted image candidate and having a certain extent in a left-right direction from the center; A step of superimposing the image candidate with the predetermined region masked on an image region of the camera image from which the image candidate was extracted; 13. An image processing method comprising:

Images