Arithmetic device and auto calibration restoration method

Abstract

To achieve simple restoration of automatic calibration.SOLUTION: A calculation device according to an embodiment includes a controller having a function of restoring automatic calibration of an in-vehicle camera. When receiving an error notification of the auto calibration executed based on a detection result of a lane from an image captured by the in-vehicle camera during traveling of a vehicle, the controller determines a point where a recognition degree in image recognition of the lane is high based on operation record data of the vehicle, and calculates a reference position for the auto calibration from an image captured by the in-vehicle camera at the point where the recognition degree is high, when a degree of divergence between the calculated reference position and an existing value that is a value being set for the reference position is equal to or greater than a threshold value, the existing value is updated with the calculated reference position.SELECTED DRAWING: Figure 6

Description

[Technical Field]

[0001] The disclosed embodiments relate to a computing device and an auto-calibration repair method. [Background technology]

[0002] Conventionally, when detecting inter-vehicle distances or lanes by image recognition of images taken by an in-vehicle camera such as a drive recorder, information on the mounting angle of the in-vehicle camera is required. The mounting angle can basically be determined from the mounting height, lateral position, horizon position, center position of the driving lane, and bonnet position captured in the image, which are set when the in-vehicle camera is installed.

[0003] However, there is a problem that the accuracy is low if the settings are only made when the vehicle is stopped. For this reason, there is a technology that uses image recognition of images taken by the in-vehicle camera while the vehicle is actually moving to detect lanes, and then automatically calibrates the in-vehicle camera's installation setting information based on the detection results. This technology is sometimes called "auto-calibration."

[0004] In auto-calibration, the range in which the lane is likely to be detected is calculated using the aforementioned horizon position, the center position of the driving lane, and the hood position as reference positions, and image recognition is performed within that range.

[0005] However, the mounting angle of an in-vehicle camera may shift from the initial mounting angle due to unexpected contact, aging, etc. If the mounting angle shifts, the image recognition range of the auto-calibration based on the reference position described above also shifts, which may result in an auto-calibration error.

[0006] In addition, a technology has been proposed to detect misalignment in the mounting angle of an on-board camera by comparing the area of ​​the bumper reflected in the image when there is a misalignment and when there is no misalignment (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0007] [Patent Document 1] Japanese Patent Application Laid-Open No. 2016-078484 Summary of the Invention [Problem to be solved by the invention]

[0008] However, the above-mentioned conventional technology does not take auto-calibration into consideration. In other words, although the technology disclosed in Patent Document 1 can detect deviations in the mounting angle of the vehicle-mounted camera, it does not have an algorithm for calculating the range in which it is estimated that lane marks are easily detected, and therefore cannot be applied as is when repairing auto-calibration.

[0009] One aspect of the embodiment has been made in view of the above, and aims to provide a calculation device and an auto-calibration restoration method that can realize simple auto-calibration restoration. [Means for solving the problem]

[0010] According to one aspect of the embodiment, a computing device includes a controller having a function of repairing auto-calibration of an on-board camera. When the controller receives an error notification for the auto-calibration performed based on lane detection results from images captured by the on-board camera while the vehicle is traveling, the controller determines a point with a high degree of recognition in image recognition of the lane based on operation record data of the vehicle, calculates a reference position for the auto-calibration from images captured by the on-board camera at the point with a high degree of recognition, and, when a deviation between the calculated reference position and an existing value that is currently set for the reference position is equal to or greater than a threshold, updates the existing value with the calculated reference position. [Effects of the Invention]

[0011] According to one aspect of the embodiment, when an auto-calibration error occurs, the controller automatically calculates the reference position from an image of a point where it is estimated that a lane is likely to be detected. The controller also determines the degree of deviation between the calculated reference position and an existing value being set, and if the degree of deviation is large, automatically updates the reference position. In other words, according to one aspect of the embodiment, it is possible to easily repair the auto-calibration. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a diagram illustrating an outline of an auto-calibration restoration method according to an embodiment. [Figure 2] FIG. 2 is a diagram illustrating an example of the attachment setting information. [Figure 3] FIG. 3 is an explanatory diagram of the horizon position, the hood position, the center position between lanes, and the image recognition range during auto-calibration. [Figure 4] FIG. 4 is a diagram illustrating an example of the configuration of an auto-calibration and restoration system. [Figure 5] FIG. 5 is a diagram illustrating an example of the configuration of a drive recorder. [Figure 6] FIG. 6 is a diagram illustrating an example of the configuration of the center device. [Figure 7] FIG. 7 is a diagram (part 1) showing a processing sequence according to the first embodiment. [Figure 8] FIG. 8 is a diagram (part 2) showing a processing sequence according to the first embodiment. [Figure 9] FIG. 9 is a diagram showing an example of a confirmation screen for installation setting information. [Figure 10] FIG. 10 is a diagram showing an example of a setting change operation. [Figure 11] FIG. 11 is a diagram showing a processing sequence according to the second embodiment. [Figure 12] FIG. 12 is an explanatory diagram of steps S305 to S307. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, embodiments of the arithmetic device and auto-calibration repair method disclosed in the present application will be described in detail with reference to the accompanying drawings. However, the present invention is not limited to the following embodiments.

[0014] In the following description, the arithmetic device according to the embodiment is assumed to be the center device 100 (see FIG. 1). The auto-calibration and restoration method according to the embodiment is assumed to be an auto-calibration and restoration method executed by the controller 103 (see FIG. 6) of the center device 100.

[0015] In the following description, the in-vehicle device according to the embodiment is a drive recorder 10 (see FIG. 1). The drive recorder 10 is a communication type that is capable of communicating with a center device 100. In the following description, the auto-calibration restoration system according to the embodiment is an auto-calibration restoration system 1 (see FIG. 1) that includes the drive recorder 10 and the center device 100.

[0016] In the following, when it is necessary to distinguish between multiple identical elements, a number in the form "-n" (n is a natural number equal to or greater than 1) may be added after the symbol indicating the element. When there is no particular need to distinguish between them, this numbering will not be used.

[0017] In the following description, the term "auto-calibration" will be referred to as "auto-calib" where appropriate. In the following description, the terms "predetermined," "specific," and "fixed" may be read as "predetermined."

[0018] First, an overview of the auto-calibration repair method according to the embodiment will be described with reference to Fig. 1. Fig. 1 is an explanatory diagram of the auto-calibration repair method according to the embodiment. Fig. 2 is a diagram showing an example of the installation setting information 14a. Fig. 3 is an explanatory diagram of the horizon position, hood position, lane center position, and image recognition range during auto-calibration.

[0019] As shown in FIG. 1, the auto-calibration and recovery system 1 includes one or more drive recorders 10 and a center device 100.

[0020] The drive recorder 10 is a video recording device mounted on a vehicle. The drive recorder 10 according to the embodiment includes a camera 12a. The camera 12a is provided so as to be able to capture at least an external image of the area in front of the vehicle.

[0021] The drive recorder 10 is installed while the vehicle is stopped, for example, in a parking lot or a dealer's service shop. At the time of installation, installation setting information 14a is manually set. As shown in Fig. 2, the installation setting information 14a includes the installation position height, lateral position, horizon position, hood position, and lane center position.

[0022] The horizon position, hood position, and lane center position are set on the image as shown in FIG. 3. The horizon position is set to a position that literally corresponds to the horizon. The hood position is set to the highest position of the hood that appears in the image captured by camera 12a. The lane center position is set to the center position between two lanes that will be recognized by image recognition, i.e., the center position of the driving lane. The lateral position is a position that is shifted toward the driver's seat from the center position in the vehicle width direction.

[0023] The set installation setting information 14a is sent to the center device 100 (not shown), and is linked to the vehicle ID, which is the identification information of the vehicle in which the drive recorder 10 that sent the information is installed, and registered in the installation setting information DB (Database) 102d (see Figure 6) of the center device 100.

[0024] Note that the installation setting information 14a is set while the vehicle is stopped at the time of installation, and therefore, there is a risk that the accuracy will be low if the vehicle is stopped in an underground parking lot, on a slope, etc. Therefore, the drive recorder 10 executes auto-calibration.

[0025] In autocalib, the drive recorder 10 calculates an image recognition range R1 in which it is estimated that lanes are easily detected using the horizon position, the hood position, and the center position between lanes as reference positions, and detects lanes by performing image recognition within the image recognition range R1. As shown in Fig. 3, the drive recorder 10 calculates the image recognition range R1 as, for example, a trapezoidal range between the horizon position and the hood position that is larger than the driving lane.

[0026] Returning to the explanation of Figure 1, the installed drive recorder 10 executes an operation record by recording a certain period of operation record data 14b, including images of the exterior of the vehicle captured by the camera 12a, in a ring buffer memory in an overwritable manner while the vehicle is running. The certain period is, for example, 24 hours. In addition to the images of the exterior of the vehicle, the operation record data 14b includes at least the date and time, position information (latitude and longitude), direction (heading), and vehicle speed.

[0027] In addition, while recording the driving operation, the drive recorder 10 performs image recognition processing on the outside-of-vehicle video using an AI model for image recognition. The image recognition processing is also performed during auto-calibration. The AI ​​model is, for example, a DNN (Deep Neural Network) model trained using a machine learning algorithm. This AI model is trained in advance so that it can recognize the type, position, color, etc. of each object shown in the outside-of-vehicle video. Each object shown in the outside-of-vehicle video includes a lane that will be detected during auto-calibration.

[0028] The drive recorder 10 is also configured to be able to detect specific events. The drive recorder 10 is configured to be able to detect various specific events, such as the occurrence of an accident, a near miss, reaching a predetermined position, ignoring a traffic light, etc., based on, for example, the results of image recognition, changes in vehicle speed, changes in G-value, changes in latitude and longitude, etc.

[0029] When the drive recorder 10 detects a specific event, it sets the operation record data for a certain period of time before and after the detection time to be overwritten-protected. Alternatively, the drive recorder 10 records the operation record data for a certain period of time before and after the detection time to a separate recording medium. This overwriting protection setting or recording to a separate recording medium may be performed in response to an instruction from the center device 100.

[0030] Furthermore, the drive recorder 10 transmits the recorded operation record data 14b to the center device 100 periodically or in real time. The drive recorder 10 may transmit only the operation record data 14b that has been set to be overwritten to the center device 100 only when a specific event is detected. The operation record data 14b transmitted to the center device 100 is associated with the above-mentioned vehicle ID and stored in the operation record data DB 102a (see FIG. 6) of the center device 100.

[0031] Furthermore, the drive recorder 10 periodically (for example, for each trip) performs the above-described auto-calibration in parallel with recording the driving operation. It is desirable that auto-calibration be performed when the vehicle is traveling at a certain speed or above and is traveling straight on a road with two or more lanes, so that the lane can be easily detected.

[0032] Incidentally, the mounting angle of the drive recorder 10 may shift from the initial mounting angle due to unexpected contact, aging, etc. If the mounting angle shifts, the image recognition range R1 described above also shifts, which may result in an auto-calibration error.

[0033] Therefore, in this embodiment, when an auto-calibration error occurs, the controller 103 of the center device 100 determines a point where lane recognition is high, calculates a reference position for auto-calibration from the camera image of the point where recognition is high, and determines the degree of deviation from the existing value (step S3).

[0034] Specifically, it is assumed that the drive recorder 10 executes auto-calibration and an error occurs (step S1), and then the drive recorder 10 transmits an auto-calibration error notification to the center device 100 (step S2).

[0035] When the center device 100 receives this auto-calibration error notification, the controller 103 executes a "level of recognition determination process" in step S3. In the level of recognition determination process, the controller 103 extracts operation record data 14b of the vehicle in question from the operation record data DB 102a after the date and time the error occurred, and analyzes the extracted operation record data 14b in chronological order.

[0036] Then, the controller 103 extracts straight driving locations that are a certain distance or more (for example, 100 m) based on the analysis results. The controller 103 then determines whether the locations are flat and have two or more lanes. This allows the controller 103 to determine whether the locations are suitable for calculating the image recognition range R1 where it is estimated that lanes are easily detected.

[0037] Next, the controller 103 executes an "image recognition process." Here, the controller 103 uses an AI model to perform image recognition on the image included in the driving record data 14b for the relevant location, and detects the lane and the hood.

[0038] Next, the controller 103 executes a "reference position calculation process." In the reference position calculation process, the controller 103 calculates the intersection point (point at infinity) of the detected lanes and the hood position. Then, the controller 103 calculates the position of a horizontal line passing through the calculated intersection point as the horizon position, and the position of a vertical line as the center position between the lanes.

[0039] Next, the controller 103 executes a "deviation determination process." In the deviation determination process, the controller 103 calculates the error between the calculated reference position and an existing value, and if the error is equal to or greater than a threshold, determines that the deviation is large and that the reference position needs to be updated.

[0040] Then, the controller 103 updates the reference position (step S4) and transmits the updated reference position to the drive recorder 10 (step S5). The drive recorder 10 reflects the updated reference position in the setting of the mounting setting information 14a.

[0041] As a result, even if the mounting angle of the drive recorder 10 is misaligned and an auto-calibration error occurs, the controller 103 of the center device 100 can automatically change the settings of the mounting setting information 14a for the vehicle in question and reflect the changes in the drive recorder 10. Therefore, according to the auto-calibration restoration method of the embodiment, it is possible to restore auto-calibration when an auto-calibration error occurs.

[0042] 1, an example in which the controller 103 automatically performs the restoration has been given, but it is also possible to configure the center device 100 so that an administrator can easily change the settings manually. Below, a configuration example of the auto-calibration restoration system 1 will be described in more detail, with the details of this manual method being described as a first embodiment. Also, the details of the automatic method described with reference to FIG. 1 will be described as a second embodiment.

[0043] First Embodiment 4 is a diagram showing an example of the configuration of the auto-calibration and restoration system 1. As shown in FIG. 4, the auto-calibration and restoration system 1 includes drive recorders 10-1, 10-2, . . . 10-m (m is a natural number equal to or greater than 3) and a center device 100.

[0044] Each drive recorder 10 and the center device 100 are connected to each other so as to be able to communicate with each other via a network N1, which may be the Internet, a mobile phone network, a C-V2X (Cellular Vehicle to Everything) communication network, or the like.

[0045] As described above, the drive recorder 10 executes operation recording and records a certain period of operation record data 14b in a ring buffer memory in an overwritable manner. Also, as described above, the drive recorder 10 is configured to be able to detect various specific events in parallel with the operation recording.

[0046] When the drive recorder 10 detects a specific event, it sets the operation record data 14b for a certain period of time before and after the detection time to be overwritten-prohibited. The drive recorder 10 also transmits the recorded operation record data 14b to the center device 100 periodically or in real time. Alternatively, the drive recorder 10 transmits only the operation record data 14b that has been set to be overwritten-prohibited to the center device 100 only when the drive recorder 10 detects a specific event.

[0047] Furthermore, the drive recorder 10 periodically (for example, for each trip) executes the above-described auto-calibration in parallel with recording the driving operation. If an auto-calibration error occurs, the drive recorder 10 transmits an auto-calibration error notification to the center device 100.

[0048] The center device 100 is realized, for example, as a private cloud. The center device 100 may also be realized as a public cloud. The center device 100 is managed, for example, by a business operator that operates a data center that includes the center device 100. The center device 100 collects and stores the driving record data 14b transmitted from each drive recorder 10.

[0049] Furthermore, when the center device 100 receives an auto-calibration error from the drive recorder 10, it manually or automatically changes the settings of the installation setting information 14a of the vehicle. In the automatic mode, the center device 100 executes the information processing described with reference to FIG.

[0050] In the manual method, the center device 100 provides a confirmation screen for the existing installation setting information 14a of the vehicle to the administrator of the center device 100. The center device 100 also provides the administrator of the center device 100 with a user interface that enables the administrator to easily change the settings of the installation setting information 14a via the confirmation screen. The specific contents will be described later using FIGS. 9 and 10.

[0051] Next, a description will be given of an example of the configuration of the drive recorder 10. Fig. 5 is a diagram showing an example of the configuration of the drive recorder 10. As shown in Fig. 5, the drive recorder 10 has a communication unit 11, a sensor unit 12, an HMI (Human Machine Interface) unit 13, a storage unit 14, and a controller 15.

[0052] The communication unit 11 is realized by a network adapter etc. The communication unit 11 is wirelessly connected to the network N1, and transmits and receives information to and from the center device 100 via the network N1.

[0053] The sensor unit 12 is a group of various sensors mounted on the drive recorder 10. The sensor unit 12 includes, for example, a camera 12a, a G sensor 12b, a GPS (Global Positioning System) sensor 12c, and a vehicle speed sensor 12d.

[0054] As described above, the camera 12a is provided so as to be able to capture at least an external image in front of the vehicle. The camera 12a is installed near the windshield, the dashboard, etc. The camera 12a may also be installed near the rear window, etc., so as to be able to capture an image behind the vehicle.

[0055] The G sensor 12b measures acceleration (G) applied to the drive recorder 10. The GPS sensor 12c measures the GPS position (latitude and longitude) of the vehicle. Note that the sensor unit 12 may include various sensors other than the camera 12a, the G sensor 12b, the GPS sensor 12c, and the vehicle speed sensor 12d.

[0056] In addition to the sensor unit 12, the drive recorder 10 is also connected to an in-vehicle sensor 5, which is a group of various sensors mounted on the vehicle. The in-vehicle sensor 5 includes, for example, an accelerator sensor and a brake sensor. The in-vehicle sensor 5 is connected to the drive recorder 10 via an in-vehicle network such as a CAN (Controller Area Network).

[0057] The HMI unit 13 is a component that provides interface components related to input and output to the driver or the like who uses the drive recorder 10. The HMI unit 13 includes an input interface that accepts input operations from the driver or the like. The HMI unit 13 also includes an output interface that presents visual information and audio information to the driver or the like.

[0058] For example, the HMI unit 13 includes a display and a speaker. The display is realized by, for example, a touch panel display. The touch panel display corresponds to the input interface and output interface described above. The touch panel display displays, for example, operation components for operating the drive recorder 10.

[0059] Note that these operation components may not be displayed as software components on the touch panel display, but may be provided as hardware components in the HMI unit 13. The speaker corresponds to an output interface that outputs guidance voices and the like from the drive recorder 10.

[0060] The storage unit 14 is realized by a storage device such as a read-only memory (ROM), a random access memory (RAM), a flash memory, etc. In the example of Fig. 5, the storage unit 14 stores attachment setting information 14a, operation record data 14b, and an image recognition model 14c.

[0061] The installation setting information 14a and the operation record data 14b have already been explained, and therefore will not be explained here. The image recognition model 14c corresponds to the AI ​​model described above. The image recognition model 14c operates as an image recognition AI when loaded into the controller 15. The image recognition model 14c loaded into the controller 15 is configured to be able to recognize the type, position, color, etc. of each of the above-mentioned objects shown in each frame when each frame of the outside-of-vehicle video is input.

[0062] The controller 15 corresponds to a so-called processor. The controller 15 is realized by a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a GPU (Graphical Processing Unit), or the like. The controller 15 executes a program according to an embodiment (not shown) stored in the storage unit 14, using RAM as a work area. The controller 15 can also be realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0063] The controller 15 executes information processing according to the processing sequences shown in Figures 7, 8 and 11. The explanation using Figures 7, 8 and 11 will be given later.

[0064] Next, a description will be given of an example of the configuration of the center device 100. Fig. 6 is a diagram showing an example of the configuration of the center device 100. As shown in Fig. 6, the center device 100 includes a communication unit 101, a storage unit 102, and a controller 103.

[0065] The center device 100 is also connected to an HMI unit 50. The HMI unit 50 is a component that provides interface components related to input and output to a manager or the like who uses the center device 100. The HMI unit 50 is similar to the HMI unit 13 described above, and therefore a detailed description thereof will be omitted here.

[0066] The communication unit 101 is realized by a network adapter etc. The communication unit 101 is connected to the network N1 by wire or wirelessly, and transmits and receives information to and from the drive recorder 10 via the network N1.

[0067] The storage unit 102 is realized by a storage device such as a ROM, a RAM, a flash memory, an HDD (Hard Disk Drive), etc. In the example of Fig. 6, the storage unit 102 stores an operation record data DB (Database) 102a, a map information DB 102b, an image recognition model 102c, and an installation setting information DB 102d.

[0068] The operation record data DB 102a is a database that stores operation record data 14b collected from each drive recorder 10. The operation record data 14b collected from each drive recorder 10 is associated with the vehicle ID of the vehicle that sent the data and stored in the operation record data DB 102a.

[0069] The map information DB 102b is a database of map information used in generating a confirmation screen, which will be described later. The image recognition model 102c is an AI model for image recognition used in the image recognition process of step S3 described above. The function is the same as the image recognition model 14c of the drive recorder 10 described above, but since the center device 100 usually has higher processing performance than the drive recorder 10, the center device 100 may be provided so as to enable image recognition with higher accuracy than the image recognition model 14c.

[0070] The installation setting information DB 102d is a database that stores each piece of installation setting information 14a transmitted from each drive recorder 10. The installation setting information 14a transmitted from each drive recorder 10 is associated with the vehicle ID of the vehicle that transmitted the information and stored in the installation setting information DB 102d.

[0071] The controller 103 corresponds to a so-called processor. The controller 103 is realized by a CPU, an MPU, a GPU, or the like. The controller 103 executes a program according to an embodiment (not shown) stored in the storage unit 102, using a RAM as a work area. The controller 103 can also be realized by an integrated circuit such as an ASIC or an FPGA.

[0072] The controller 103, like the above-described controller 15, executes information processing according to the processing sequences shown in FIGS.

[0073] Next, these processing sequences will be described. Note that the description using FIG. 11 will be given later as the second embodiment. FIG. 7 is a diagram (part 1) showing the processing sequence according to the first embodiment. FIG. 8 is a diagram (part 2) showing the processing sequence according to the first embodiment.

[0074] 7, the drive recorder 10 is first installed and set up (step S101). At this time, the installation setting information 14a is manually set while the vehicle is stopped. After the setting, the controller 15 of the drive recorder 10 transmits the installation setting information 14a to the center device 100 (step S102).

[0075] When the controller 103 of the center device 100 receives the installation setting information 14a, it registers it in the installation setting information DB 102d (step S103).

[0076] The controller 15 of the drive recorder then executes operation recording while the vehicle is running (step S104). The controller 15 also determines whether a transmission trigger for transmitting the operation record data 14b has been detected (step S105). The transmission trigger may be, for example, when the transmission is periodic, at the arrival of the periodic timing; or, for example, when the transmission is real-time, at any time; or, for example, when the transmission is upon detection of a specific event, at the time of detection.

[0077] If a transmission trigger is detected (step S105, Yes), the controller 15 transmits the operation record data 14b to the center device 100 (step S106). When the controller 103 of the center device 100 receives the operation record data 14b, it stores the operation record data in the operation record data DB 102a (step S107).

[0078] If the transmission trigger is not detected (step S105, No), the controller 15 of the drive recorder 10 determines whether or not an auto-calibration event is detected (step S108). An auto-calibration event occurs when, for example, the period for executing auto-calibration arrives, the vehicle speed is equal to or greater than a certain value, and the vehicle is traveling straight on a road with two or more lanes.

[0079] If an auto-calibration event is detected (step S108, Yes), the controller 15 executes auto-calibration (step S109). If an auto-calibration event is not detected (step S108, No), the controller 15 repeats the process from step S104.

[0080] After executing autocalibration, the controller 15 determines whether an autocalibration error has occurred (step S110). If an autocalibration error has occurred (step S110, Yes), the controller 15 transmits an autocalibration error notification to the center device 100 (step S111) and repeats the processing from step S104.

[0081] When the controller 103 of the center device 100 receives the auto-calibration error notification, it associates the notification with the vehicle ID of the vehicle and stores it in the installation setting information DB 102d (step S112). If an auto-calibration error has not occurred (step S110, No), the controller 15 repeats the process from step S104.

[0082] The processing sequence described with reference to FIG. 7 is a processing sequence common to both embodiments, which is also executed in a second embodiment described later.

[0083] Next, Fig. 8 shows a processing sequence that is executed after the auto-calibration error notification is sent to the center device 100 in step S111 of Fig. 7. In the manual method, as shown in Fig. 8, when the controller 103 of the center device 100 receives the auto-calibration error notification, it notifies the administrator of the center device 100 (step S201). Such notification is performed by sending an email, displaying on a screen, etc.

[0084] Next, the controller 103 extracts the vehicle operation record data 14b and the installation setting information 14a of the vehicle based on the operation of the notified administrator (step S202), and then generates and displays a confirmation screen for the installation setting information 14a (step S203).

[0085] An example of the confirmation screen and an example of how to operate it will now be described. Fig. 9 is a diagram showing an example of the confirmation screen for the installation setting information 14a. Fig. 10 is a diagram showing an example of a setting change operation.

[0086] 9, the confirmation screen for the installation setting information 14a has an area R11, an area R12, and an area R13. This confirmation screen visualizes the driving route of the vehicle after the date and time when the autocalib error occurred and the currently set (existing) installation setting information 14a, allowing the administrator to view them. In addition, this confirmation screen is provided so that it can accept an operation to change the settings of the installation setting information 14a from the administrator.

[0087] Area R11 displays a field for specifying the display period. Area R12 displays map information showing the driving route of the vehicle. Area R13 displays images included in the driving record data 14b and reference positions (horizon position, hood position, center position between lanes) based on the installation setting information 14a. Area R13 also has a setting reflect button B1. The setting reflect button B1 is an operation button for confirming the contents of the setting changes.

[0088] By the administrator specifying the start date and time of the display period in area R11 and operating the update button, the vehicle's travel route T1 is superimposed on the map information in area R12. In addition, when the administrator specifies any one point on the travel route T1, an image corresponding to that point is displayed in area R13. A reference position based on the installation setting information 14a being set is superimposed on the image. This allows the administrator to easily check the image taken at any one point on the vehicle's travel route T1 and the installation setting information 14a corresponding to this image.

[0089] To change the settings of the installation setting information 14a, it is desirable to specify a location with a high degree of recognition where lanes are easily detected in AutoCalib's image recognition. Therefore, the administrator checks the map information for area R12 and specifies a location that is flat, has two or more lanes, and is a straight-line driving area of ​​a certain distance or more. In the example of Figure 9, the administrator uses cursor C1 to specify point P1 as a location that meets these conditions.

[0090] Area R13 displays the image of point P1 and the reference position based on the installation setting information 14a being set. As shown in Figure 10, the administrator can individually move the horizon position, hood position, and lane center position to any desired position by, for example, using a drag-and-drop operation with cursor C1. This allows the administrator to easily change the reference position settings by simply moving the visualized reference positions relative to the image while checking the image.

[0091] After changing the settings, the administrator can confirm and reflect the changed settings by operating the setting reflect button B1. When the setting reflect button B1 is operated, the changed settings are reflected in the installation setting information 102d of the center device 100. At the same time, the changed settings are transmitted to the drive recorder 10 and reflected in the installation setting information 14a of the drive recorder 10.

[0092] In Figures 9 and 10, the cursor C1 is represented by a finger, which indicates that if the HMI unit 50 of the center device 100 is a touch panel display, the administrator can perform touch operations with their finger.

[0093] Returning to the explanation of Fig. 8, when the controller 103 receives a setting change operation from the administrator via the confirmation screen (step S204, Yes), it determines whether or not a setting reflection operation has been performed (step S205). If there is no setting change operation (step S204, No), the controller 103 repeats step S204. Also, if there is no setting reflection operation (step S205, No), the controller 103 repeats the processing from step S204.

[0094] If the setting reflection operation is performed (step S205, Yes), the controller 103 reflects the setting content in the installation setting information DB 102d (step S206). Furthermore, the controller 103 transmits the updated reference position after the setting reflection to the drive recorder 10 (step S207).

[0095] When the controller 15 of the drive recorder 10 receives the updated reference position, the controller 15 reflects the received updated reference position in the mounting setting information 14a (step S208).

[0096] As described above, the center device 100 (corresponding to an example of a "computing device") according to the first embodiment includes the controller 103 having a function of repairing the auto-calibration of the camera 12a (corresponding to an example of an "on-board camera"). When the controller 103 receives an error notification of the auto-calibration performed based on the lane detection result from an image captured by the camera 12a while the vehicle is traveling, the controller 103 generates and displays a confirmation screen for the installation setting information 14a based on the installation setting information 14a of the camera 12a. Furthermore, when the controller 103 receives an operation to change the reference position for the auto-calibration included in the installation setting information 14a from an administrator (corresponding to an example of a "user") via the confirmation screen, the controller 103 updates the existing value, which is the value currently set for the reference position, with the reference position changed by the change operation.

[0097] Therefore, according to the first embodiment, when an auto-calibration error occurs, the controller 103 generates and displays a confirmation screen for the installation setting information 14a, visualizes the (existing) installation setting information 14a being set for the administrator, and allows the administrator to view it. Furthermore, if the controller 103 receives an operation to change the reference position from the administrator via the confirmation screen, it updates the existing value with the reference position changed by the change operation. In other words, according to the first embodiment, it is possible to realize a simple auto-calibration repair.

[0098] <Second embodiment> Next, a processing sequence according to the second embodiment will be described with reference to Fig. 11. Fig. 11 is a diagram showing a processing sequence according to the second embodiment.

[0099] As described above, the processing sequence explained with reference to Fig. 7 is also executed in the second embodiment. Fig. 11 shows the processing sequence executed after the auto-calibration error notification is transmitted to the center device 100 in step S111 of Fig. 7.

[0100] In the case of the automatic method, as shown in FIG. 11, when the controller 103 of the center device 100 receives the auto-calibration error notification, it starts automatic analysis for changing the settings of the installation setting information 14a of the vehicle (step S301).

[0101] In the automatic analysis, the controller 103 extracts operation record data 14b from the date and time when the error occurred (step S302). Then, the controller 103 analyzes the extracted operation record data 14b in chronological order and extracts straight driving locations for a certain distance or more (step S303).

[0102] Then, the controller 103 determines whether the extracted corresponding location is flat and has two or more lanes (step S304). If the conditions are not met (step S304, No), the controller 103 repeats the process from step S303.

[0103] If the relevant location is flat and has two or more lanes (Yes at step S304), the controller 103 performs image recognition of the relevant location and detects the lanes and the hood (step S305).

[0104] Then, the controller 103 calculates the intersection point (point at infinity) of the detected lanes and the position of the hood (step S306).The controller 103 then calculates the position of a horizontal line passing through the intersection point calculated in step S306 as the horizon position and the position of a vertical line passing through the intersection point as the center position between the lanes (step S307).

[0105] A specific example of steps S305 to S307 is shown in Fig. 12. Fig. 12 is an explanatory diagram of steps S305 to S307. As shown in the upper part of Fig. 12, in step S305, the controller 103 detects lanes L1, L2, and hood H1 by image recognition.

[0106] Then, as shown in the middle part of FIG. 12, the controller 103 calculates an intersection point P2 of the lane L1 and the lane L2 in step S306. The controller 103 also calculates the highest position of the hood H1 as the hood position. The hood H1 corresponds to an example of a "vehicle body." This allows the controller 103 to calculate a reference position based on the lane L1, lane L2, and hood H1 detected by image recognition.

[0107] 12, in step S307, controller 103 calculates the position of the horizontal line passing through intersection point P2 as the horizon position, and the position of the vertical line as the center position between lanes. This makes it possible to calculate the horizon position and the center position between lanes with high accuracy from intersection point P2, which is a point at infinity based on lanes L1 and L2.

[0108] Returning to the description of Fig. 11, the controller 103 then determines whether the errors between the calculated horizon position, hood position, and lane center position and the existing values ​​(currently being set) are equal to or greater than thresholds (step S308).

[0109] If the reference position is equal to or greater than the threshold value (step S308, Yes), the controller 103 reflects the reference position calculated in steps S306 and S307 in the installation setting information DB 102d (step S309). Furthermore, the controller 103 transmits the updated reference position reflected in the installation setting information DB 102d to the drive recorder 10 (step S310).

[0110] When the controller 15 of the drive recorder 10 receives the updated reference position, the controller 15 reflects the received updated reference position in the mounting setting information 14a (step S311).

[0111] If the value is less than the threshold value (No at step S308), the controller 103 does not reflect the reference positions calculated at steps S306 and S307 in the mounting setting information DB 102d, and does not transmit the reference positions to the drive recorder 10.

[0112] As described above, the center device 100 (corresponding to an example of a "computing device") according to the second embodiment includes the controller 103 having a function of repairing the auto-calibration of the camera 12a (corresponding to an example of an "on-board camera"). When the controller 103 receives an error notification for the auto-calibration executed based on the lane detection result from the image captured by the camera 12a while the vehicle is traveling, the controller 103 determines a point with a high degree of recognition in the image recognition of the lane based on the operation record data 14b of the vehicle. The controller 103 also calculates a reference position for the auto-calibration from the image captured by the camera 12a at the point with a high degree of recognition. When the error (corresponding to an example of a "deviation") between the calculated reference position and an existing value that is currently set for the reference position is equal to or greater than a threshold, the controller 103 updates the existing value with the calculated reference position.

[0113] Therefore, according to the second embodiment, when an auto-calibration error occurs, the controller 103 automatically calculates the reference position from an image of a point where it is estimated that lanes are likely to be detected. Then, the controller 103 determines the degree of deviation between the calculated reference position and the existing value being set, and if the degree of deviation is large, the controller 103 automatically updates the reference position. In other words, according to the second embodiment, it is possible to easily repair the auto-calibration.

[0114] In the above-described embodiments, the center device 100 is an example of a computing device that executes the auto-calibration restoration method manually or automatically, but the drive recorder 10 may also be the computing device. That is, the auto-calibration restoration method may be executed only on the vehicle side without going through the center device 100. In this case, the computing device is not limited to the drive recorder 10, and may be any on-board device such as a car navigation device or an ECU (Electronic Control Unit). In addition, when the auto-calibration restoration method is executed only on the vehicle side, it is desirable that the on-board device that executes the auto-calibration restoration method has sufficient processing capabilities for image recognition, etc.

[0115] In the first embodiment described above, an example of a confirmation screen for the installation setting information 14a is shown in FIG. 9, but this is merely an example, and various information not shown in FIG. 9 may be displayed. For example, not only the travel route T1 but also a graph of speed change on the travel route T1 may be displayed. When an arbitrary point on the time series of this graph is designated, an image corresponding to that point may be displayed in the region R13. In addition, information such as the date and time when the auto-calibration error occurred and the vehicle ID of the vehicle may be displayed.

[0116] Further advantages and modifications will readily occur to those skilled in the art. Therefore, the invention in its broader aspects is not limited to the specific details and representative embodiments shown and described above. Accordingly, various modifications may be made without departing from the spirit or scope of the general inventive concept as defined by the appended claims and their equivalents. [Explanation of symbols]

[0117] 1. Auto-calibration repair system 5. In-vehicle sensors 10 Drive Recorder 11 Communications Department 12 Sensor section 12a Camera 13 HMI section 14 Storage section 15 Controller 50 HMI section 100 Center Device 101 Communications Department 102 Storage section 103 Controller

Claims

1. A controller having a function of restoring auto-calibration of an in-vehicle camera, The controller When an error notification is received for the auto-calibration executed based on the lane detection result from the image captured by the on-board camera while the vehicle is traveling, a point with a high degree of recognition in the image recognition of the lane is determined based on the operation record data of the vehicle, calculating a reference position for the auto-calibration from an image captured by the vehicle-mounted camera at the point with high recognition rate; updating the existing value with the calculated reference position when the degree of deviation between the calculated reference position and an existing value that is being set for the reference position is equal to or greater than a threshold value; Computing device.

2. The controller extracting from the driving record data a location where the vehicle is traveling straight ahead for a predetermined distance or longer; If the location is flat and has two or more lanes, the location is determined to be a point with a high degree of recognition. The computing device of claim 1 .

3. The controller Detecting the vehicle body and the lane in the image captured by the on-board camera at the point with high recognition rate by image recognition; calculating the reference position based on the detected vehicle body and the detected lane; The computing device according to claim 2 .

4. the reference positions include at least a horizon position, a vehicle body position, and a center position between the lanes; The controller Calculating the vehicle body position based on the detected vehicle body; calculating the horizon position and the center position between the lanes based on the detected lanes; The computing device according to claim 3 .

5. The controller Calculating the intersection of the detected lanes; The position of a horizontal line passing through the calculated intersection is set as the horizon position, and the position of a vertical line passing through the intersection is set as the center position between the lanes. The computing device according to claim 4.

6. A controller having a function of restoring auto-calibration of an in-vehicle camera, The controller When an error notification is received for the auto-calibration executed based on the lane detection result from the image captured by the in-vehicle camera while the vehicle is traveling, a confirmation screen for the installation setting information is generated and displayed based on the installation setting information of the in-vehicle camera; When a change operation of the reference position for the auto-calibration included in the mounting setting information is received from the user via the confirmation screen, an existing value that is a value being set for the reference position is updated with the reference position changed by the change operation. Computing device.

7. The controller generating the confirmation screen so that the reference position is superimposed on the image captured by the vehicle-mounted camera; The computing device according to claim 6.

8. The controller generating the confirmation screen so that a user can arbitrarily move the reference position displayed superimposed on the image captured by the vehicle-mounted camera; The computing device according to claim 7.

9. The controller a driving route of the vehicle is displayed superimposed on map information based on the driving record data of the vehicle, and when a user designates any one point on the driving route, the confirmation screen is generated so that an image captured by the on-board camera at the designated point is displayed; 9. The computing device according to claim 7 or 8.

10. 1. A controller-implemented auto-calibration repair method, comprising: When an error notification is received for auto-calibration of the on-board camera, which is executed based on the result of lane detection from an image captured by the on-board camera while the vehicle is traveling, determining a point with a high degree of recognition in image recognition of the lane based on operation record data of the vehicle; calculating a reference position for the auto-calibration from an image captured by the vehicle-mounted camera at the point with high recognition rate; updating the existing value with the calculated reference position when a deviation between the calculated reference position and an existing value that is being set for the reference position is equal to or greater than a threshold value; Auto-calibration repair method including:

Images