Image processing device, image processing system, and image processing method

Abstract

To provide an image processing device, an image processing system, and an image processing method which reproduce an actual visual field of a user.SOLUTION: A drive recorder 10 comprises, an infrared LED 3, a camera 5, a velocity sensor 7, a communication unit 11, a storage unit 12, and a control unit 13. The infrared camera is capable of capturing an infrared image of a user's face irradiated with infrared rays. The image processing device has a control unit to be mounted on a mobile body. The control unit detects user's visual field estimation information and generates a visual field image corresponding to the visual field estimation information based on the image captured by the camera.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The disclosed embodiments relate to an image processing device, an image processing system, and an image processing method. [Background technology]

[0002] Conventionally, drive recorders and the like often simply record images taken by a camera capturing images of the outside of the vehicle. However, the actual field of view of the user driving the vehicle changes depending on factors such as the direction of the user's face, the direction of their line of sight, and whether their eyelids are open or closed, so there is a discrepancy between the image captured by the camera and the actual field of view.

[0003] For this reason, situations may occur that are captured in the image but not visible to the user, and it is not easy to identify that such a situation occurred when investigating the cause of an accident after the fact using footage recorded by a drive recorder.

[0004] There is a technology that detects biological information related to the user's visual field, such as blinking, and converts a video signal based on the level of alertness estimated from the blinking (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Laid-Open No. 2007-003618 Summary of the Invention [Problem to be solved by the invention]

[0006] However, the above-mentioned conventional technology has room for further improvement in terms of reproducing the user's actual field of view and making it easier to identify the cause of an accident, for example. The above-mentioned conventional technology merely attempts to adjust the user's circadian rhythm by changing the blue light emission intensity based on the user's level of alertness.

[0007] One aspect of the embodiment has been made in consideration of the above, and aims to provide an image processing device, an image processing system, and an image processing method that can reproduce the user's actual field of view. [Means for solving the problem]

[0008] According to one aspect of the embodiment, an image processing device includes a control unit, the control unit acquiring an image of a vehicle traveling direction and information on a driver's eyelid state, The area outside the field of view of the driver, which is identified according to the state of the driver's eyelids, is displayed in the image. A masked mask image is generated. [Effects of the Invention]

[0009] According to one aspect of the embodiment, the actual field of view of the user can be reproduced. [Brief explanation of the drawings]

[0010] [Figure 1] FIG. 1 is a diagram (part 1) outlining an image processing method according to an embodiment. [Figure 2] FIG. 2 is a diagram (part 2) outlining the image processing method according to the embodiment. [Figure 3] FIG. 3 is a diagram (part 3) outlining the image processing method according to the embodiment. [Figure 4] FIG. 4 is a diagram illustrating an example of the configuration of an image processing system according to the embodiment. [Figure 5] FIG. 5 is a block diagram showing an example of the configuration of the drive recorder according to the embodiment. [Figure 6] FIG. 6 is a diagram showing map information of the field of view range relative to a reference frame having vehicle speed and user age as dimensions. [Figure 7] FIG. 7 is a graph of the map information of FIG. [Figure 8] FIG. 8 is an explanatory diagram (part 1) of the detection process executed by the detection unit. [Figure 9] FIG. 9 is an explanatory diagram (part 2) of the detection process executed by the detection unit. [Figure 10]FIG. 10 is an explanatory diagram (part 1) of the generation process executed by the generation unit. [Figure 11] FIG. 11 is an explanatory diagram (part 2) of the generation process executed by the generation unit. [Figure 12] FIG. 12 is an explanatory diagram (part 3) of the generation process executed by the generation unit. [Figure 13] FIG. 13 is an explanatory diagram (part 4) of the generation process executed by the generation unit. [Figure 14] FIG. 14 is a flowchart showing a processing procedure executed by the drive recorder according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0011] Hereinafter, embodiments of an image processing device, an image processing system, and an image processing method disclosed in the present application will be described in detail with reference to the accompanying drawings. Note that the present invention is not limited to the embodiments described below.

[0012] In the following description, the user's "visual field status" is information for estimating the user's visual field. The visual field status corresponds to an example of "visual field estimation information." The visual field estimation information includes the user's facial orientation, gaze direction, eyelid opening / closing state, speed, age, etc.

[0013] First, an overview of an image processing method according to an embodiment will be described with reference to Figs. 1 to 3. Fig. 1 is a diagram (part 1) outlining the image processing method according to an embodiment. Fig. 2 is a diagram (part 2) outlining the image processing method according to an embodiment. Fig. 3 is a diagram (part 3) outlining the image processing method according to an embodiment.

[0014] In the following description, it is assumed that the image processing device according to the embodiment is a drive recorder 10 mounted on a vehicle V, as shown in Fig. 1. The drive recorder 10 has an in-camera 5a and an out-camera 5b.

[0015] The in-camera 5a is provided so as to be able to capture an image of the interior of the vehicle V. The out-camera 5b is provided so as to be able to capture an image of the exterior of the vehicle V.

[0016] 2, the drive recorder 10 also has an infrared LED (light emitting diode) 3. The infrared LED 3 is provided, for example, near the in-camera 5a.

[0017] Then, the drive recorder 10 uses the infrared LED 3 and the in-camera 5a to detect the visual field state of the user D. As shown in Fig. 2, the visual field state includes the face direction of the user D, the line of sight direction, and whether the eyelids are open or closed.

[0018] In this embodiment, the case where the line of sight direction is detected using the infrared LED 3 is taken as an example, but this is merely an example, and the line of sight direction may be detected based only on the image captured by the in-camera 5a.

[0019] Then, the drive recorder 10 generates a visual field image according to the visual field conditions based on the image captured by the outer camera 5b so that the visual field of the user D is reproduced according to the detected visual field conditions.

[0020] Specifically, as shown in the upper part of FIG. 3, the drive recorder 10 first superimposes a reference frame BR, which indicates the visual field visible to a 20-year-old person at a speed of 0 km / h, on the image captured by the outer camera 5b, using the position corresponding to the center of the line of sight of the user D when looking straight ahead as a reference. The position corresponding to the center of the line of sight of the user D when looking straight ahead may be predetermined as a fixed value, such as the center of the image captured by the outer camera 5b. Alternatively, the reference frame BR may be adjustable depending on the position of the user D's eyes or the installation position of the drive recorder 10. As shown in the same figure, the area outside the reference frame BR may be processed, such as by being semi-transparent, to reduce visibility compared to the image captured by the outer camera 5b.

[0021] Then, as shown in the lower part of FIG. 3 , the drive recorder 10 changes the position of the reference frame BR to a position based on the position corresponding to the center of the gaze detected as the visual field status of user D. The center of the gaze is specifically set as follows: First, the drive recorder 10 detects the head orientation and pupil position of user D as the visual field status of user D. The drive recorder 10 detects the gaze direction of user D from the detected head orientation and pupil position of user D. The drive recorder 10 sets the position on the captured image corresponding to the detected gaze direction of user D when viewed from the position of the outer camera 5b as the center of the gaze. At this time, correction may be made taking into account the position difference between the position of the outer camera 5b and the position of user D's eyes. The drive recorder 10 also performs processing to superimpose mask areas MR1 ​​and MR2 corresponding to the open / closed state of the eyelids, which are also detected as the visual field status, on the reference frame BR.

[0022] The mask areas MR1 ​​and MR2 are areas that, like the outside of the reference frame BR, are processed to reduce visibility compared to the image captured by the outer camera 5b, such as by making them semi-transparent. The mask areas MR1 ​​and MR2 are examples of "covering." The mask area MR1 corresponds to the upper eyelid of the user D, and the mask area MR2 corresponds to the lower eyelid of the user D.

[0023] In this manner, an image obtained by superimposing the reference frame BR and the mask ranges MR1 ​​and MR2 on the image captured by the outer camera 5b to narrow down the actual field of view range RR of the user D is referred to as a "field of view image" in this embodiment. That is, the image processing method according to the embodiment is an image processing method executed by the drive recorder 10 mounted on the vehicle V, which detects the field of view conditions of the user D and generates a field of view image according to the field of view conditions based on the image captured by the outer camera 5b.

[0024] The drive recorder 10 generates such a field of view image for each frame of the image captured by the outer camera 5b, and can continuously record while repeatedly overwriting the video data made up of such field of view images.

[0025] In addition, when the drive recorder 10 detects an impact at the time of an accident as an event, for example, using an acceleration sensor mounted on the vehicle V, it can record video data consisting of field of view images for a certain period of time before and after the time of detection using event recording, which is non-volatile recording that cannot be overwritten.

[0026] Furthermore, the drive recorder 10 can transmit the recorded video data to a server device as needed.

[0027] This allows the actual field of view of user D to be reproduced, making it easier to identify the cause of an accident, for example.

[0028] Additionally, the image processing method according to the embodiment can create a visual field image according to the speed of the vehicle V and the age of the user D. It is generally known that the visual field of the user D changes depending on the speed of the vehicle V and aging.

[0029] In the image processing method according to the embodiment, the speed of the vehicle V is determined based on the publicly known document "The effect of reducing the driving speed of automobiles on reducing traffic accidents" ([online], Cabinet Office, [searched December 17, 2021], Internet<URL:https: / / www8.cao.go.jp / koutu / taisaku / max-speed / k_3 / pdf / s5.pdf> ), a first calculation formula was created to calculate the rate at which the field of view of user D decreases in response to an increase in the speed of vehicle V. The first calculation formula can be expressed as "Y = 0.0071 × 2 - 2.0553X + 177.9", where X is the speed of vehicle V and Y is the field of view relative to reference frame BR.

[0030] Regarding aging, the publicly available material "The Relationship between Field of View and Safe Driving for the Elderly" ([online], National Police Agency, [Retrieved December 17, 2021], Internet<URL:https: / / www.npa.go.jp / koutsuu / kikaku / koureiunten / kaigi / 3 / siryoh / shiryo9.pdf> ), a second calculation formula was created to calculate the rate at which user D's visual field range decreases with age. The second calculation formula can be expressed as "y = -64.6x + 15338" where x is user D's age and y is the visual field range relative to the reference frame BR.

[0031] Then, in the image processing method according to the embodiment, map information of the field of view range relative to the reference frame BR, which has the speed of the vehicle V and the age of the user D as dimensions, is created based on these two calculation formulas, and a field of view image is generated based on this map information. Details of the map information will be described later using FIGS. 6 and 7.

[0032] An example of the configuration of the image processing system 1 including the drive recorder 10 to which the image processing method according to the above-described embodiment is applied will be described in more detail below.

[0033] 4 is a diagram showing an example of the configuration of an image processing system 1 according to an embodiment. As shown in FIG. 4, the image processing system 1 includes one or more drive recorders 10 and a server device 100.

[0034] Each drive recorder 10 and the server device 100 are connected to each other so as to be able to communicate with each other via a network N such as the Internet or a mobile phone network.

[0035] The drive recorder 10 is an example of an image processing device, and as described above, detects the visual field conditions of the user D and generates a visual field image according to the visual field conditions based on the image captured by the outer camera 5b.

[0036] The server device 100 is a device that receives, stores, and manages the video data transmitted from each drive recorder 10. Furthermore, the server device 100 can perform analysis for accident investigation and create educational data for driving instruction, etc., based on the received video data.

[0037] Next, Fig. 5 is a block diagram showing an example of the configuration of the drive recorder 10 according to the embodiment. Note that Fig. 5 shows only the components necessary to explain the features of this embodiment, and omits the description of general components.

[0038] In other words, the components shown in Figure 5 are conceptual functional components and do not necessarily have to be physically configured as shown. For example, the specific form of distribution and integration of each block is not limited to that shown in the figure, and all or part of them can be functionally or physically distributed and integrated in any unit depending on various loads, usage conditions, etc.

[0039] In addition, in the description using FIG. 5, the description of components that have already been described may be simplified or omitted.

[0040] As shown in FIG. 5, the drive recorder 10 according to the embodiment includes an infrared LED 3, a camera 5, a speed sensor 7, a communication unit 11, a storage unit 12, and a control unit 13.

[0041] The infrared LED 3 irradiates the face of the user D with infrared light. The camera 5 includes the above-mentioned in-camera 5a and out-camera 5b. The in-camera 5a includes an infrared camera. The infrared camera is provided so as to be able to capture an infrared image of the face of the user D irradiated with infrared light.

[0042] In this embodiment, the camera 5 includes the inner camera 5a and the outer camera 5b, but the camera 5 may be configured as a 360-degree camera.

[0043] The speed sensor 7 is connected to the drive recorder 10 via a CAN (Controller Area Network) or the like. The speed sensor 7 measures the speed of the vehicle V.

[0044] The communication unit 11 is realized by, for example, a network adapter etc. The communication unit 11 is connected to the network N wirelessly, and transmits and receives information to and from the server device 100 via the network N.

[0045] The memory unit 12 is realized by a memory device such as a RAM (Random Access Memory) or a flash memory, and in the example of Figure 5, it stores an image recognition AI 12a, user information 12b, field of view image information 12c, and recorded video information 12d.

[0046] The image recognition AI 12a is an AI model for image recognition. Specifically, the image recognition AI 12a is a DNN (Deep Neural Network) model trained using a machine learning algorithm. The image recognition AI 12a is read into a detection unit 13a (described later) as a DNN model, and is then configured to be able to distinguish various objects contained in an image captured by a camera 5 when the image is input to the detection unit 13a. Furthermore, the image recognition AI 12a is further configured to be able to distinguish the facial orientation of the user D when the user D is identified as an object.

[0047] The user information 12b is information relating to the attributes of the user D. The user information 12b includes, for example, the age of the user D. The user information 12b is set in advance, for example, when the vehicle V is purchased. Furthermore, if the vehicle V is a rental car or a shared car, the user information 12b is set based on authentication information when the user D is authenticated.

[0048] The field of view image information 12c is information related to the generation of a field of view image. The field of view image information 12c includes the position and size of the specified reference frame BR as well as the map information described above. FIG. 6 is a diagram showing map information of the field of view range relative to the reference frame BR, with the speed of the vehicle V and the age of the user D as dimensions. FIG. 7 is a graph of the map information of FIG. 6.

[0049] As shown in Figures 6 and 7, if the visual field visible to a 20-year-old person at a speed of 0 km / h, i.e., the visual field indicated by the specified reference frame BR, is taken as 100%, it can be seen that the visual field of user D gradually narrows as the speed of vehicle V increases or as user D's age increases.

[0050] The generation unit 13b of the control unit 13, which will be described later, generates a field of view image based on the speed of the vehicle V, the age of the user D, and such map information.

[0051] Returning to the explanation of Fig. 5, the recorded video information 12d is information including a group of recorded videos captured by the camera 5 and recorded by a video recording unit 13c of the control unit 13, which will be described later.

[0052] The control unit 13 is a controller, and is realized, for example, by a CPU (Central Processing Unit) or an MPU (Micro Processing Unit) using RAM as a work area to execute various programs stored in a storage device inside the drive recorder 10. The control unit 13 can also be realized, for example, by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0053] The control unit 13 has a detection unit 13a, a generation unit 13b, a video recording unit 13c, and a transmission unit 13d, and realizes or executes the functions and actions of information processing described below.

[0054] The detector 13a detects the visual field of the user D based on the image captured by the in-camera 5a. The detector 13a controls the infrared LED 3 in relation to the detection of the line of sight of the user D.

[0055] Fig. 8 is an explanatory diagram (part 1) of the detection process executed by the detection unit 13a, and Fig. 9 is an explanatory diagram (part 2) of the detection process executed by the detection unit 13a.

[0056] The detection unit 13a acquires a captured image of the user D captured by the in-camera 5a and inputs the captured image to the image recognition AI 12a. Furthermore, the detection unit 13a acquires an image recognition result output from the image recognition AI 12a as a result of inputting the captured image. Furthermore, the detection unit 13a detects the orientation of the user D's face based on the image recognition result acquired from the image recognition AI 12a.

[0057] As shown in Figure 8, the detection unit 13a detects the direction of user D's face based on the image recognition result by assuming that user D's median plane is the XZ plane and estimating the roll angle, pitch angle, and yaw angle about three orthogonal axes with the center of user D's head as the origin.

[0058] As shown in FIG. 9, the detection unit 13a detects the gaze direction of the user D based on the positional relationship between the pupil e1 of the user D in the infrared image captured by the infrared camera and the infrared illumination reflection image e2 that appears on the eyeball, in addition to the direction of the face.

[0059] Although the example in which the detection unit 13a detects the gaze direction using an infrared image has been given here, the gaze direction may be detected only using an image captured by a visible light camera without using an infrared image. In such a case, the detection unit 13a uses the position of, for example, the inner corner or outer corner of the eye of the user D captured in the captured image as a reference point, and detects the gaze direction based on the position of the iris relative to the reference point.

[0060] Furthermore, the detection unit 13a detects the open / closed states of the upper eyelid e31 and the lower eyelid e32 based on the image recognition results.

[0061] Returning to the explanation of Fig. 5, the detection unit 13a outputs the detected visual field condition of the user D to the generation unit 13b. The generation unit 13b generates a visual field image according to the visual field condition detected by the detection unit 13a, based on the image captured by the outer camera 5b. The generation unit 13b generates the visual field image based on the detection result of the detection unit 13a, the speed of the vehicle V measured by the speed sensor 7, the user information 12b, and the visual field image information 12c.

[0062] Fig. 10 is an explanatory diagram (part 1) of the generation process executed by the generation unit 13b. Fig. 11 is an explanatory diagram (part 2) of the generation process executed by the generation unit 13b. Fig. 12 is an explanatory diagram (part 3) of the generation process executed by the generation unit 13b. Fig. 13 is an explanatory diagram (part 4) of the generation process executed by the generation unit 13b.

[0063] 10, the generation unit 13b superimposes a reference frame BR on the image captured by the outer camera 5b. As already mentioned, the specified reference frame BR is based on a position corresponding to the center of the line of sight of the user D when looking straight ahead, and is the visual field range visible to a 20-year-old person at a speed of 0 km / h.

[0064] 11, the generation unit 13b superimposes a mask area MR1 corresponding to the upper eyelid e31 and a mask area MR2 corresponding to the lower eyelid e32 within the reference frame BR in accordance with the eyelid open / close state detected by the detection unit 13a as the visual field state of the user D. At this time, the generation unit 13b determines the size of the mask area MR1 in accordance with the degree of closure of the upper eyelid e31. The generation unit 13b also determines the size of the mask area MR2 in accordance with the degree of closure of the lower eyelid e32. In other words, the generation unit 13b changes the sizes of the mask areas MR1 ​​and MR2 in accordance with the eyelid open / close state.

[0065] Although FIG. 11 shows an example of rectangular mask areas MR1 ​​and MR2, they may have a shape that includes a curve according to the shape of the eyelid.

[0066] 11 shows an example in which the mask areas MR1 ​​and MR2 are made semi-transparent, but they may be made mosaic-processed, for example, as long as they are less visible than the image captured by the outer camera 5b. The mask areas MR1 ​​and MR2 may also be completely filled in.

[0067] Therefore, a state in which the eyelids are closed due to blinking, dozing, or the like may be represented by the reference frame BR being completely filled in black, as shown in FIG.

[0068] 10 to 12 to a position based on the position corresponding to the center point of the line of sight detected by the detection unit 13a as the visual field state of the user D (see the lower part of FIG. 3). This makes it possible to record, as an image, a change in the visual field of the user D in response to the movement of the line of sight of the user D.

[0069] Furthermore, the generation unit 13b adjusts the reference frame BR in accordance with the speed of the vehicle V and the age of the user D based on the map information included in the visual field image information 12c.

[0070] At this time, the generation unit 13b processes the specified reference frame BR so that the actual field of view range RR of the user D, which has narrowed due to an increase in the speed of the vehicle V and the aging of the user D, is clearly displayed inside the reference frame BR, as shown in Fig. 13. Note that it is preferable that the frame line RR1 of the actual field of view range RR and the reference frame BR are drawn so as to be clearly distinguishable from each other using different colors or the like.

[0071] Furthermore, it is preferable that the area outside the actual field of view RR be processed to make it less visible than the image captured by the outer camera 5b, such as by making it semi-transparent, as has been done in the past.

[0072] 10 to 12, the generation unit 13b may adjust the reference frame BR according to the speed of the vehicle V and the age of the user D. In this case, the generation unit 13b may clearly indicate the actual field of view range RR inside the reference frame BR by a frame line RR1 as shown in FIG. 13, or may narrow the reference frame BR itself according to the speed of the vehicle V and the age of the user D without drawing the frame line RR1.

[0073] Returning to the explanation of Fig. 5, generation unit 13b outputs the generated field of view image to video recording unit 13c. In the case of continuous recording, video recording unit 13c records video data consisting of the field of view image generated by generation unit 13b in recorded video information 12d while repeatedly overwriting it.

[0074] Furthermore, when the video recording unit 13c detects an impact at the time of an accident as an event, for example, by an acceleration sensor (not shown), the video recording unit 13c records the video data consisting of the field of view image generated by the generation unit 13b in the recorded video information 12d as an event recording. Note that the event recording is not limited to when an accident occurs, but may also be performed as appropriate, for example, when a sudden stop occurs, when dangerous driving is detected, or in response to an operation by the user D.

[0075] Furthermore, the video recording unit 13c causes the transmitting unit 13d to transmit video data consisting of the field of view image to the server device 100. The video recording unit 13c causes the server device 100 to transmit, for example, video data that is the subject of event recording.

[0076] The transmitting unit 13d transmits the video data corresponding to the instruction to the server device 100 via the communication unit 11 based on the instruction from the video recording unit 13c.

[0077] The server device 100 performs analysis processing for, for example, accident investigation based on the video data consisting of the field of view images received from the drive recorder 10. The server device 100 also uses the received video data for, for example, creating teaching material data for driving instruction.

[0078] Next, a processing procedure executed by the drive recorder 10 according to the embodiment will be described with reference to Fig. 14. Fig. 14 is a flowchart showing the processing procedure executed by the drive recorder 10 according to the embodiment.

[0079] 14, the detection unit 13a detects the visual field state of the user D (step S101). Then, the generation unit 13b generates a visual field image according to the visual field state detected by the detection unit 13a (step S102).

[0080] Then, the video recording unit 13c records or transmits the video consisting of the field of view image (step S103), and the process ends. Note that while the drive recorder 10 is running, the control unit 13 repeats the flow of Fig. 14 (steps S101 to S103).

[0081] As described above, the drive recorder 10 (corresponding to an example of an "image processing device") according to the embodiment is an image processing device having a control unit 13 mounted on a vehicle V (corresponding to an example of a "mobile body"). The control unit 13 detects the visual field condition of the user D (corresponding to an example of "visual field estimation information") and generates a visual field image according to the visual field condition based on the image captured by the camera 5.

[0082] Therefore, according to the drive recorder 10 according to the embodiment, the actual field of view of the user D can be reproduced, making it easier to identify the cause of an accident, for example.

[0083] Furthermore, the control unit 13 generates the visual field image by superimposing mask areas MR1 ​​and MR2 (corresponding to an example of a "cover") corresponding to the eyelids of the user D on the captured image.

[0084] Therefore, the drive recorder 10 according to the embodiment can reproduce the field of view of the user D according to the eyelids.

[0085] The visual field conditions include whether the eyelids of the user D are open or closed, and the control unit 13 changes the sizes of the mask areas MR1 ​​and MR2 depending on whether the eyelids of the user D are open or closed.

[0086] Therefore, the drive recorder 10 according to the embodiment can reproduce the field of view of the user D that changes depending on the movement of the eyelids.

[0087] The control unit 13 also generates the visual field image in which a reference frame BR indicating the visual field range visible to a 20-year-old person when the speed of the vehicle V is zero is superimposed at a predetermined position.

[0088] Therefore, the drive recorder 10 according to the embodiment can reproduce the field of view of the user D relative to the reference frame BR.

[0089] The visual field conditions include the facial orientation and gaze direction of the user D, and the control unit 13 generates the visual field image by changing the position of the reference frame BR to be superimposed based on the facial orientation and gaze direction of the user D.

[0090] Therefore, the drive recorder 10 according to the embodiment can reproduce the field of view of the user D relative to the reference frame BR, which changes depending on the face orientation and line of sight direction.

[0091] Furthermore, the control unit 13 superimposes the mask areas MR1 ​​and MR2 within the reference frame BR.

[0092] Therefore, the drive recorder 10 according to the embodiment can reproduce the actual field of view of the user D according to the open / closed state of the eyelids within the reference frame BR.

[0093] In addition, the field of view conditions include the speed of the vehicle V and the age of the user D, and the control unit 13 narrows the field of view range of the user D in the captured image as the speed of the vehicle V increases or as the age of the user D increases to generate the field of view image.

[0094] Therefore, according to the drive recorder 10 according to the embodiment, the actual field of view of the user D according to the speed of the vehicle V and the age of the user D can be reproduced.

[0095] Furthermore, the control unit 13 performs processing to reduce the visibility of areas other than the visual field range RR of the user D in the visual field image.

[0096] Therefore, according to the drive recorder 10 according to the embodiment, the actual visual field range RR of the user D can be clearly displayed relative to the other areas.

[0097] Moreover, the image processing system 1 according to the embodiment includes a drive recorder 10 mounted on a vehicle V, and a server device 100. The drive recorder 10 detects the visual field conditions of a user D, generates a visual field image according to the visual field conditions based on an image captured by a camera 5, and transmits the generated visual field image to the server device 100. The server device 100 receives the visual field image from the drive recorder 10, and uses the received visual field image at least for analyzing the cause of an accident or creating teaching material data for driving instruction.

[0098] Therefore, according to the image processing system 1 according to the embodiment, the actual field of view of the user D can be reproduced, making it easier to identify the cause of an accident, for example.

[0099] In addition, the image processing method of the embodiment is an image processing method executed by a drive recorder 10 mounted on a vehicle V, and includes detecting the visual field conditions of a user D and generating a visual field image corresponding to the above visual field conditions based on an image captured by a camera 5.

[0100] Therefore, according to the image processing method of the embodiment, the actual field of view of the user D can be reproduced, making it easier to identify the cause of an accident, for example.

[0101] In the above-described embodiment, the image processing device is mounted on the vehicle V, but it may be mounted on a moving body other than the vehicle V, such as a ship or an airplane.

[0102] Further advantages and modifications will readily occur to those skilled in the art. Therefore, the invention in its broader aspects is not limited to the specific details and representative embodiments shown and described above. Accordingly, various modifications may be made without departing from the spirit or scope of the general inventive concept as defined by the appended claims and their equivalents. [Explanation of symbols]

[0103] 1. Image processing system 3 infrared LEDs 5. Camera 5a Front camera 5b Rear camera 7 Speed ​​Sensor 10 Drive Recorder 11 Communications Department 12 Storage section 12a Image recognition AI 12b User Information 12c Field of view image information 12d Recorded Video Information 13 Control Unit 13a Detection unit 13b Generator 13c Video Recording Section 13d Transmitter 100 Server device BR Frame of Reference D User MR1,MR2 mask range RR Actual Field of View V vehicle

Claims

1. An image processing device having a control unit, The control unit Obtaining video of the vehicle's direction of travel and information on the driver's eyelid condition, generating a masked image in which an area outside the field of view of the driver, which is identified according to the state of the driver's eyelids, is masked in the image; Image processing device.

2. The control unit changing a mask range according to an open / closed state of the driver's eyelids; The image processing device according to claim 1 .

3. At least a part of the mask range has a curved shape corresponding to the shape of the eyelid. The image processing device according to claim 2 .

4. The control unit A reference frame, which is a specified range visible to a person of a predetermined reference age at a predetermined reference speed, is superimposed on the image, and an area within the reference frame that is outside the field of view of the driver identified according to the state of the driver's eyelids is masked.

4. The image processing device according to claim 1.

5. The control unit changing the position of the reference frame based on the direction of the driver's face and line of sight; The image processing device according to claim 4 .

6. The control unit acquiring speed information of the vehicle; The higher the speed, the narrower the frame of reference.

6. The image processing device according to claim 4 or 5.

7. The control unit obtaining age information of the driver; The older the driver is, the narrower the reference frame is.

6. The image processing device according to claim 4 or 5.

8. An image processing system including an image processing device and a server, The image processing device includes: Transmitting an image of the vehicle's traveling direction and information about the driver's eyelid state to the server; The server generating a masked image in which an area outside the field of view of the driver, which is identified according to the state of the driver's eyelids, is masked in the image; Image processing system.

9. An image processing method performed by an image processing device, Obtaining video of the vehicle's direction of travel and information on the driver's eyelid condition, generating a masked image in which an area outside the field of view of the driver, which is identified according to the state of the driver's eyelids, is masked in the image; Image processing methods.

10. An image processing device having a control unit, The control unit A reference frame, which is a prescribed visual field range visible to a person of a predetermined reference age at a predetermined reference speed, is displayed on an image captured in the direction of travel of the vehicle, while processing the display outside the reference frame; acquiring information regarding the open / closed state of the driver's eyelids, and when it is determined that the driver's eyelids are closed, processing the display of the entire image; Image processing device.

11. The control unit The image is processed to make the outside of the reference frame semi-transparent; When it is determined that the eyelids of the driver are closed, the entire image is made semi-transparent. The image processing device according to claim 10.

12. The control unit The reference frame is narrowed as the speed of the vehicle increases. The image processing device according to claim 10 or 11.

13. The control unit changing a position of the reference frame in the image in response to a change in the driver's line of sight; The image processing device according to any one of claims 10 to 12.

14. The control unit When the eyelids of the driver are closed for a predetermined period of time or more, it is determined that the eyelids of the driver are closed. The image processing device according to any one of claims 10 to 13.

Images