Program, in-vehicle monitoring method, and in-vehicle monitoring device

Abstract

Conventional in-vehicle monitoring systems made it difficult for the driver or passenger to properly view images of the person being monitored inside the vehicle. [Solution] An in-vehicle monitor device 1 comprises a receiving unit 14 that receives images of the interior of a vehicle transmitted by one or more camera devices 2 that photograph the rear seat or cargo area of ​​a vehicle, a display unit 171 that displays the images received by the receiving unit 14, a position acquisition unit 161 that acquires the current position at two or more different points in time, a speed acquisition unit 162 that acquires the speed of movement using the two or more current positions acquired by the position acquisition unit 161, and a display control unit 163 that changes the display of the image according to the speed of movement. This allows the driver, passenger, etc. to appropriately view images of the person being monitored inside the vehicle.

Description

Technical Field

[0001] The present invention relates to a program for an in-vehicle monitor that receives and outputs images from a camera that photographs the rear seat or cargo area of an automobile.

Background Art

[0002] Conventionally, there has been a dedicated monitor that can check the condition of a child in the rear seat during driving, and a monitor that is set on the dashboard (see Non-Patent Document 1).

[0003] In addition, there has been a robot having a function of preventing a child abandonment accident (see Non-Patent Document 2).

Prior Art Documents

Non-Patent Documents

[0004]

Non-Patent Document 1

Non-Patent Document 2

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, with conventional technology, the driver, passengers in the front seat, etc., could not properly view the images taken of the person being monitored inside the vehicle.

[0006] Specifically, with conventional technology, it was dangerous to view images of the subject being monitored even while driving at high speed.

[0007] Furthermore, in conventional technology, it was sometimes impossible to automatically adjust the screen brightness appropriately according to the ambient light, time of day, and driving area, making it difficult to properly understand the condition of the person being monitored within the image.

[0008] Furthermore, with conventional technology, it was not possible to alert the driver or passenger if the monitored person's posture became unstable.

[0009] Furthermore, in conventional technology, it was not possible to implement a function to prevent children from being left unattended in cars using, for example, a mobile device (e.g., a smartphone) without using equipment such as robots.

[0010] Furthermore, in conventional technology, it was not possible to perform operations on the display device side to make the received image of the monitored subject easier to view.

[0011] Furthermore, in conventional technology, only the voice of the driver or other person is output to the child, so it was not possible to show the child an image of the driver or other person and operate the car in a way that would make the child feel sufficiently safe. [Means for solving the problem]

[0012] The in-vehicle monitoring device of the first invention comprises a receiving unit that receives images of the interior of a vehicle transmitted by one or more camera devices that photograph the rear seat or cargo area of ​​a vehicle; a display unit that displays the images received by the receiving unit; a position acquisition unit that acquires the current position at two or more different points in time; a speed acquisition unit that uses the acquired position information to acquire the speed of movement; and a display control unit that changes the display of the image according to the speed of movement.

[0013] With this configuration, the driver and passenger can appropriately view images of the person being monitored inside the vehicle, depending on the vehicle's speed.

[0014] Furthermore, the in-vehicle monitoring device of this second invention is further equipped with an environment acquisition unit that acquires environmental information, and the display control unit changes the brightness of the screen on which the image is displayed according to the environmental information.

[0015] With this configuration, depending on the environment, the driver, passenger, or other person can view the captured images of the person being monitored inside the vehicle at an appropriate brightness.

[0016] Furthermore, the in-vehicle monitoring device of this third invention further comprises, compared to the second invention, an external brightness management unit that stores external brightness information that identifies the brightness outside in association with two or more pieces of position information, and an acquisition determination unit that determines whether or not the environment acquisition unit can acquire illuminance. The environment acquisition unit acquires illuminance if it can acquire it, and if it cannot acquire illuminance, it acquires position information corresponding to the time and current location. The display control unit changes the brightness of the screen on which the image is displayed according to the illuminance when the environment acquisition unit has acquired illuminance, and acquires external brightness information associated with position information corresponding to the time and current location when the environment acquisition unit cannot acquire illuminance, and changes the brightness of the screen on which the image is displayed according to the external brightness information.

[0017] With this configuration, even when illumination cannot be obtained, the captured images can be viewed by the driver, passenger, or others in the front seat at an appropriate brightness according to the ambient light, allowing them to see the person being monitored inside the vehicle.

[0018] Further, the in-vehicle monitor device of the fourth invention, with respect to the first invention, further includes a state acquisition unit that analyzes the image received by the reception unit and acquires state information for identifying the state of the object to be monitored in the rear seat or the luggage compartment, a state determination unit that determines whether the state information acquired by the state acquisition unit satisfies the attention-calling condition, and an attention-calling unit that outputs attention-calling information when the state determination unit determines that the attention-calling condition is satisfied.

[0019] With such a configuration, it is possible to analyze an image of the object to be monitored in the vehicle and call the driver, the person in the passenger seat, etc. to attention according to the acquired state of the object to be monitored.

[0020] Further, the in-vehicle monitor device of the fifth invention, with respect to the fourth invention, is an in-vehicle monitor device in which the state determination unit compares the state information acquired by the state acquisition unit with reference state information and determines whether the attention-calling condition is satisfied.

[0021] With such a configuration, it is possible to analyze an image of the object to be monitored in the vehicle and call the driver, the person in the passenger seat, etc. to attention according to the result of comparing the acquired state of the object to be monitored with the reference state.

[0022] Further, the in-vehicle monitor device of the sixth invention, with respect to the first invention, further includes a leaving determination unit that determines whether a leaving condition including the inability to communicate with the camera device is met, and an attention-calling unit that outputs attention-calling information when the leaving determination unit determines that the leaving condition is met.

[0023] With such a configuration, it is possible to prevent leaving the protected person alone in the vehicle.

[0024] Further, the in-vehicle monitor device of the seventh invention, with respect to the sixth invention, further includes an image recognition unit that recognizes a person in the image received by the reception unit, and the leaving condition is the inability to communicate with the camera device in a situation where the image recognition unit recognizes a person.

[0025] With such a configuration, it is possible to appropriately prevent leaving the protected person alone in the vehicle.

[0026] In addition, the in-vehicle monitor device of the eighth invention further includes an operation reception unit that receives an operation on the image displayed by the display unit with respect to the first invention, and the display control unit is an in-vehicle monitor device that changes the display of the image according to the operation.

[0027] With such a configuration, the screen can be operated to appropriately view the situation of the protected person.

[0028] In addition, the in-vehicle monitor device of the ninth invention further includes a first imaging unit that acquires a first image of the driver or a person in the passenger seat, and a first image transmission unit that transmits the first image to the camera device, with respect to the first invention.

[0029] With such a configuration, by providing an environment where the protected person can see the images of the driver and the person in the passenger seat, the protected person can be made to feel at ease even while moving.

[0030] In addition, the in-vehicle monitor device of the tenth invention further includes a first microphone that collects a first voice, a first voice transmission unit that transmits the first voice to the camera device, a first voice reception unit that receives a second voice from the camera device, and a first voice output unit that outputs the second voice, with respect to the first invention. The camera device includes a second microphone that collects the second voice, a second transmission unit that transmits the second voice, a second reception unit that receives the first voice transmitted by the in-vehicle monitor device, and a second voice output unit that outputs the first voice.

[0031] With such a configuration, by providing an environment where the protected person can clearly hear the voices of the driver and the person in the passenger seat, the protected person can be made to feel at ease even while moving.

[0032] Furthermore, the in-vehicle monitoring device of the eleventh invention is an in-vehicle monitoring device in which the receiving unit receives images from two or more camera devices, and the display unit displays two or more images received from two or more camera devices, in contrast to the first invention.

[0033] This configuration allows the driver, passenger, or other person to appropriately view images of two or more subjects being monitored inside the vehicle.

[0034] Furthermore, the in-vehicle monitoring device of the twelfth invention further comprises a camera management unit that stores a camera identifier indicating a camera to be hidden or displayed, compared to the eleventh invention, and the display control unit uses the camera identifier to display only the image corresponding to the camera to be displayed from among two or more camera devices.

[0035] With this configuration, the driver, passenger, or other relevant personnel can appropriately view only the images of the person requiring monitoring, from among the images from two or more cameras inside the vehicle.

[0036] Furthermore, the in-vehicle monitoring device of the thirteenth invention, compared to the eleventh invention, has at least two camera devices, one of which photographs the person to be protected and the other photographs the luggage, and the display control unit instructs the display unit to display the image of the person to be protected and the image of the luggage in a distinguishable manner.

[0037] This configuration allows the driver, passenger, or other person to appropriately view images of two or more protected individuals inside the vehicle, as well as images of their belongings. [Effects of the Invention]

[0038] According to the in-vehicle monitoring device of the present invention, the driver, passenger, and others can appropriately view images of the person being monitored inside the vehicle. [Brief explanation of the drawing]

[0039] [Figure 1] Conceptual diagram of the in-vehicle monitoring system A in Embodiment 1 [Figure 2] Block diagram of the in-vehicle monitoring system A. [Figure 3] Block diagram of the in-vehicle monitoring device 1. [Figure 4] Flowchart illustrating an example of operation of the in-vehicle monitoring device 1. [Figure 5] A flowchart illustrating an example of the process for obtaining the same speed. [Figure 6] A flowchart illustrating an example of the same brightness determination process. [Figure 7] A flowchart illustrating an example of the same state determination process. [Figure 8] A flowchart illustrating an example of the process for determining whether to leave the items together. [Figure 9] Figure showing an example of the same display. [Figure 10] A diagram showing an example of the same external brightness control table. [Figure 11] Figure showing an example of the same display. [Figure 12] Figure showing an example of the same display. [Figure 13] Figure showing an example of the same display. [Figure 14] Figure showing an example of the same display. [Figure 15] Block diagram of the computer system [Modes for carrying out the invention]

[0040] The embodiments of the in-vehicle monitoring device and the like will be described below with reference to the drawings. In the embodiments, components that are denoted by the same reference numerals perform the same operation, and therefore, further explanation may be omitted.

[0041] (Embodiment 1) This embodiment describes an in-vehicle monitor device that receives and outputs images from one or more cameras that photograph the rear seat or cargo area of ​​a vehicle, and a program for the in-vehicle monitor that acquires the vehicle's speed while it is moving and changes the display of the video (for example, hides it) according to the speed. The images are usually videos, but still images may also be used.

[0042] In this embodiment, a program for an in-car monitor that acquires environmental information and automatically changes the screen brightness according to that information will be described.

[0043] In this embodiment, if illuminance is obtained, the screen brightness is automatically adjusted according to that illuminance; if illuminance cannot be obtained, the screen brightness is automatically adjusted using time and location information. This section explains the program for in-car monitoring.

[0044] This embodiment describes a program for an in-car monitor that acquires the state of the monitored object and outputs warning information if the state matches the warning conditions. This embodiment describes a program for an in-car monitor that compares state information with reference state information to determine whether the warning conditions are met and outputs warning information if the state matches the warning conditions. In particular, this describes a program for an in-car monitor that detects when the monitored child's posture is poor, the child is moving rapidly, a threshold time has been exceeded, the child is not moving, or the child's face is not facing forward for a long time, and alerts the driver or passenger.

[0045] In this embodiment, a program is described that issues a warning if communication becomes impossible while a person has been recognized based on the received image, indicating the possibility of a child being left unattended.

[0046] In this embodiment, a program for an in-vehicle monitor that can perform operations on received images will be described.

[0047] In this embodiment, a program for an in-vehicle monitor that enables the monitoring party and the monitored party to mutually send and receive images and audio will be described.

[0048] This embodiment describes a program for an in-vehicle monitor that can receive images from two or more cameras and display two or more images simultaneously. The two or more cameras are, for example, installed in the rear seat and the cargo area, respectively. The two or more images are, for example, images of a single object being monitored taken from different directions (for example, the front and the side).

[0049] In this specification, information X being associated with information Y means that information Y can be obtained from information X, or information X can be obtained from information Y, and the method of association is irrelevant. Information X and information Y may be linked, may reside in the same buffer, may information X be contained in information Y, or information Y may be contained in information X, and so on.

[0050] Furthermore, in this specification, selecting or determining information Z means obtaining information Z, obtaining a pointer to information Z, obtaining the ID of information Z, setting a flag on information Z, etc., and it is sufficient to be able to access information Z.

[0051] Figure 1 is a conceptual diagram of the in-vehicle monitoring system A in this embodiment. The in-vehicle monitoring system A comprises an in-vehicle monitoring device 1 and one or more camera devices 2. In Figure 1, the reference numerals for each of the two or more camera devices 2 are 2a, 2b, 2c, and 2d. It goes without saying that the position of the camera devices 2 is not fixed. For example, considering the presence of a rear-facing child seat, a camera device 2 may be installed that has a camera pointed forward from the position of the headrest of the rear seat.

[0052] The in-vehicle monitoring device 1 is preferably a portable terminal. While it is preferable for the in-vehicle monitoring device 1 to be a smartphone, a tablet or similar device may also be used. The various functions of the in-vehicle monitoring device 1, described later, are usually implemented by a program. In other words, the in-vehicle monitoring device 1 is typically a portable terminal with an application installed that implements the various functions of the in-vehicle monitoring device 1 described later.

[0053] Figure 2 is a block diagram of the in-vehicle monitoring system A in this embodiment. Figure 3 is a block diagram of the in-vehicle monitoring device 1.

[0054] The in-vehicle monitor device 1 comprises a storage unit 11, a first imaging unit 12, a first microphone 13, a receiving unit 14, a reception unit 15, a processing unit 16, and an output unit 17. The storage unit 11 comprises an external brightness management unit 111 and a camera management unit 112. The receiving unit 14 comprises a first image receiving unit 141 and a first audio receiving unit 142. The reception unit 15 comprises an operation reception unit 151. The processing unit 16 comprises a position acquisition unit 161, a speed acquisition unit 162, a display control unit 163, an acquisition determination unit 164, an environment acquisition unit 165, a state acquisition unit 166, a state determination unit 167, an image recognition unit 168, and a placement determination unit 169. The output unit 17 comprises a display unit 171, a warning unit 172, a first image transmission unit 173, a first audio transmission unit 174, and a first audio output unit 175.

[0055] The camera device 2 includes a second imaging unit 21, a second microphone 22, a second transmission unit 23, a second image receiving unit 24, a second audio receiving unit 25, a second image display unit 26, and a second audio output unit 27.

[0056] The storage unit 11, which constitutes the in-vehicle monitor device 1, stores various types of information. These types of information include, for example, ambient brightness information and camera identifiers, which will be described later.

[0057] The ambient brightness management unit 111 stores ambient brightness information associated with one or more location information points.

[0058] Location information refers to information that identifies a location or area. Examples of location information include one or more (longitude, latitude) coordinates, a regional name, and a prefecture name.

[0059] External brightness information refers to information that identifies the brightness of the outside. External brightness information includes, for example, the brightness, lightness, and luminance of a screen. External brightness information includes sunset and sunrise times for each region, prefecture, or area indicated by two or more location information (here, for example, (latitude, longitude)), and for each season, month, week, period, or day. External brightness information also includes the outside illuminance or range of outside illuminance for each region, prefecture, or area indicated by two or more location information, and for each season, month, week, period, or day. Note that the brightness of the outside is, for example, the brightness of the image obtained by the camera built into the in-vehicle monitor device 1. More specifically, the average illuminance is calculated by summing the brightness of each pixel in the image obtained by the camera built into the in-vehicle monitor device 1 and dividing by the number of pixels.

[0060] The camera management unit 112 stores display camera information that identifies which cameras to hide or display. The display camera information consists of one or more camera identifiers that identify which cameras to hide or display.

[0061] The first camera unit 12 acquires the first image of the person watching over the subject being monitored. The person watching over is usually the driver or the person in the passenger seat.

[0062] Microphone 13 collects the first audio signal. The first audio signal is the voice of the observer.

[0063] The receiving unit 14 receives various types of information. These types of information include images of the interior of the vehicle transmitted by one or more camera devices 2 that photograph the rear seat or cargo area of ​​the vehicle, and second audio, which is audio transmitted from the camera devices 2. The images of the interior of the vehicle are usually images of the person being monitored. The person being monitored is usually a person in the back seat (for example, a child) or luggage in the cargo area. The second audio is, for example, the voice of the child being monitored or the sound of luggage falling over.

[0064] The first image receiving unit 141 receives images transmitted by one or more camera devices 2.

[0065] The first audio receiving unit 142 receives the second audio from one or more camera devices 2.

[0066] The reception unit 15 receives various instructions and information from the supervisor. These instructions and information include, for example, the operations described later.

[0067] Here, "reception" is a concept that includes receiving information entered from input devices such as touch panels, keyboards, and pens; receiving information transmitted via wired or wireless communication lines; and receiving information read from recording media such as optical discs, magnetic discs, and semiconductor memory.

[0068] The operation reception unit 151 receives operations on the image displayed by the display unit 171. These operations include, for example, zooming in, zooming out, panning, and tilting. Operations may also be received by voice. Operations (commands) received by voice are recognized by voice recognition. In such cases, the operation reception unit 151 performs voice recognition processing to acquire the operations.

[0069] The processing unit 16 performs various processes. These processes include, for example, those performed by the position acquisition unit 161, the speed acquisition unit 162, the display control unit 163, the acquisition determination unit 164, the environment acquisition unit 165, the state acquisition unit 166, the state determination unit 167, the image recognition unit 168, or the placement determination unit 169.

[0070] The position acquisition unit 161 acquires the current position at two or more points in time. The position acquisition unit 161 acquires the current position continuously over time. The current position is usually in the form of (latitude, longitude) or (latitude, longitude, altitude).

[0071] The speed acquisition unit 162 acquires the moving speed using two or more current positions acquired by the position acquisition unit 161. The process of acquiring the moving speed using two or more current positions is a known technique. The speed acquisition unit 162 usually calculates the moving speed by "moving speed = |P1 - P0| / (t1 - t0)" (where P0 is the current position at time (t0) and P1 is the current position at time (t1) (t0 < t1)).

[0072] The display control unit 163 changes the display of the image according to the moving speed acquired by the speed acquisition unit 162. For example, when the moving speed is equal to or greater than a threshold value (e.g., 10 km / h), the display control unit 163 hides the image of the monitored object. It is preferable that the display control unit 163 only hides the image of the monitored object (appropriately referred to as a camera image). In such a case, it is preferable to alert the driver or the like and display the current speed and the like. For example, when the moving speed is equal to or greater than a threshold value, the display control unit 163 transmits the image of the monitored object to the terminal of the person in the passenger seat. In such a case, the in-vehicle monitor device 1 is, for example, a terminal installed in the vehicle.

[0073] The display control unit 163 changes the brightness of the screen on which the image is displayed according to the environmental information. For example, when the illuminance is equal to or less than a threshold value (dark), the display control unit 163 reduces the brightness (Brightness) of the screen.

[0074] For example, when the environment acquisition unit 165 acquires the illuminance, the display control unit 163 changes the brightness of the screen on which the image is displayed according to the illuminance. When the environment acquisition unit 165 cannot acquire the illuminance, the display control unit 163 acquires the external brightness information associated with the position information corresponding to the current time and the current position from the external brightness management unit 111, and changes the brightness of the screen on which the image is displayed according to the external brightness information.

[0075] The display control unit 163 changes the image display in response to an operation. For example, the display control unit 163 zooms in on the image on the screen in response to a "zoom in" operation. For example, the display control unit 163 zooms out on the image on the screen in response to a "zoom out" operation. For example, the display control unit 163 pans the image on the screen in response to a "pan" operation. For example, the display control unit 163 tilts the image on the screen in response to a "tilt" operation.

[0076] The display control unit 163 uses a camera identifier to display only the image corresponding to the camera device 2 to be displayed among two or more camera devices 2. The display control unit 163 refers to the camera management unit 112 and, based on the display camera information, determines which camera device 2 to display and displays only the image from that camera device 2. This allows, for example, the driver to focus their attention on a sick child among multiple children in the back seat.

[0077] The display control unit 163 instructs the display unit 171 to display the image of the protected person and the image of the luggage in different ways so that they can be distinguished. Different ways of displaying them include, for example, different window sizes or different window frame displays (frame line color, line type, thickness, etc.).

[0078] The acquisition determination unit 164 determines whether the environment acquisition unit 165 can acquire illuminance. For example, if the in-vehicle monitor device 1 has an illuminance sensor, it can usually acquire illuminance. The determination of whether the environment acquisition unit 165 can acquire illuminance may also be made by determining whether the in-vehicle monitor device 1 has an illuminance sensor. The determination of whether the environment acquisition unit 165 can acquire illuminance may also be made by determining whether it can acquire the brightness of the image obtained by the camera built into the in-vehicle monitor device 1.

[0079] The environmental information acquisition unit 165 acquires environmental information. For example, the environmental information acquisition unit 165 acquires illuminance. For example, the environmental information acquisition unit 165 calculates brightness (average illuminance) by summing the brightness of each pixel in the image obtained by the camera built into the in-vehicle monitor device 1 and dividing by the number of pixels. The environment acquisition unit 165 acquires the current time from, for example, a clock (not shown). The environment acquisition unit 165 also acquires location information based on the current location acquired by the location acquisition unit 161. The location information may be the same as the current location. The location information may also be a regional name or prefecture name based on (latitude, longitude) or (latitude, longitude, altitude). Note that the technique of acquiring a regional name or prefecture name using map information from (latitude, longitude) or (latitude, longitude, altitude) is publicly known.

[0080] The environment acquisition unit 165 preferably acquires illuminance if it can acquire it, and if it cannot acquire illuminance, it preferably acquires the time and location information corresponding to the current location. For example, the environment acquisition unit 165 preferably acquires the time and location information corresponding to the current location when the acquisition determination unit 164 determines that it cannot acquire illuminance.

[0081] The status acquisition unit 166 analyzes the image received by the receiving unit 14 and acquires status information that identifies the status of the object being monitored in the rear seat or cargo area.

[0082] State information includes, for example, information identifying the posture of the person being monitored, the intensity of movement, the absence of movement for a threshold time or longer, and the face not facing forward for a threshold time or longer. Information identifying the posture of the person being monitored includes, for example, the coordinate values ​​of two or more specific points on the person's body, and the degree of postural distortion obtained from the coordinate values ​​of two or more specific points on the person's body. The degree of postural distortion is obtained, for example, by an increasing function with the angle of two predetermined points (e.g., the left shoulder point and the right shoulder point) relative to the horizontal as a parameter, or by a decreasing function with the similarity to the coordinate values ​​of two or more specific points on the body in a reference posture (example of reference state information) as a parameter. Note that the two predetermined points mentioned above are not limited. The two predetermined points may include, for example, the left shoulder point and the right shoulder point, or the position of the right eye and the position of the left eye.

[0083] The state determination unit 167 determines whether the state information acquired by the state acquisition unit 166 satisfies the warning conditions. The warning conditions are: "the degree of postural distortion indicated by the information identifying the posture of the person being monitored is above or greater than a threshold," "the degree indicating the intensity of movement over a predetermined period of time is above or greater than a threshold," "there is no movement for a period of time greater than the threshold," and "the face is not facing forward for a period of time greater than the threshold." The degree indicating the intensity of movement is, for example, a value obtained by a decreasing function that uses the similarity of two or more still images contained in the video as a parameter.

[0084] The state determination unit 167 compares the state information acquired by the state acquisition unit 166 with the reference state information to determine whether or not the warning conditions are met. Reference state information is information that identifies a reference (normal) state. Reference state information includes, for example, pre-set information (e.g., two or more coordinate values ​​that identify a reference posture, the degree of distortion of the reference posture, the degree indicating the intensity of the reference movement), an image taken when the vehicle starts driving, information based on the image taken when the vehicle starts driving (e.g., two or more coordinate values ​​that identify a reference posture, the degree of distortion of the reference posture, the degree indicating the intensity of the reference movement), an image taken at a previous point in time (e.g., 1 minute ago, 30 seconds ago), and information based on the image taken at a previous point in time (e.g., 1 minute ago, 30 seconds ago). Reference state information includes, for example, two or more coordinate values ​​that identify a reference posture, the degree of distortion of the reference posture, and the degree indicating the intensity of the reference movement.

[0085] The image recognition unit 168 recognizes people in the image received by the receiving unit 14. The image recognition unit 168 performs face recognition on the people in the image received by the receiving unit 14. Face recognition is just one example of person recognition.

[0086] The abandonment determination unit 169 determines whether the conditions for abandonment are met, including the fact that communication with the camera device 2 has been lost. The conditions for abandonment are those used to identify that the person being monitored has been left behind in the vehicle. For example, the conditions for abandonment are when the image recognition unit 168 has recognized a person and communication with the camera device 2 has been lost. In other words, if communication with the camera device 2 is lost after the image recognition unit 168 has stopped recognizing a person, the conditions for abandonment are not met. This is because it is assumed that the person being monitored has left the vehicle.

[0087] The output unit 17 outputs various types of information. These types of information include, for example, images of the person being monitored, and the voice or sound of the person being monitored. Here, output usually refers to display on a screen or sound output.

[0088] The display unit 171 displays the image received by the receiving unit 14. It is preferable for the display unit 171 to display two or more images received from two or more camera devices 2. The display unit 171 may output two images in picture-in-picture (PinP), but it may also output them in two windows of the same size, and the output method is not limited.

[0089] When an image of a child is displayed on the screen of the in-vehicle monitor device 1, it is preferable for the display unit 171 to show an interface for controlling the image (e.g., buttons) from hidden to visible when the screen is touched. Image controls include, for example, zooming in, zooming out, panning, and tilting.

[0090] The alerting unit 172 outputs alerting information when the status determination unit 167 determines that the alerting conditions are met. The alerting information is information intended to alert the observer. The alerting information can be, for example, sound, voice, text, light, or a combination thereof, but the type of information is not limited. The alerting information may also be status information.

[0091] The warning unit 172 preferably outputs warning information when the abandonment determination unit 169 determines that the abandonment conditions are met. The warning information here preferably comes in the form of sound. The sound may also be voice.

[0092] The first image transmission unit 173 transmits the first image acquired by the first imaging unit 12 to the camera device 2. The first image is an image of the observer.

[0093] The first audio transmission unit 174 transmits the first audio picked up by the first microphone 13 to the camera device 2. The first audio is the voice of the observer.

[0094] The first audio output unit 175 outputs the second audio received by the first audio receiving unit 142. The second audio is, for example, a child's voice, but it may also be a sound from the cargo area (for example, the sound of cargo falling over).

[0095] The second imaging unit 21, which constitutes the camera device 2, photographs the object being monitored and acquires an image.

[0096] The second microphone 22 collects the second audio. The second audio is either the voice of the person being monitored or the sound of luggage moving (e.g., luggage shifting).

[0097] The second transmitting unit 23 transmits the second audio picked up by the second microphone 22 to the in-vehicle monitoring device 1.

[0098] The second image receiving unit 24 receives the first image from the in-vehicle monitor device 1.

[0099] The second audio receiving unit 25 receives the first audio from the in-vehicle monitoring device 1.

[0100] The second image display unit 26 displays the first image received by the second image receiving unit 24. This first image is usually displayed in front of the caregiver (usually a child).

[0101] The second audio output unit 27 outputs the first audio received by the second audio receiving unit 25.

[0102] The storage unit 11, the ambient brightness management unit 111, and the camera management unit 112 are preferably made of non-volatile recording media, but can also be made of volatile recording media.

[0103] The process by which information is stored in the storage unit 11, etc. is not relevant. For example, information may be stored in the storage unit 11, etc. via a recording medium, information transmitted via a communication line, etc. may be stored in the storage unit 11, etc., or information input via an input device may be stored in the storage unit 11, etc.

[0104] The first shooting unit 12 and the second shooting unit 21 can be implemented by a camera and a camera driver. The type of camera is not limited.

[0105] The first microphone 13 and the second microphone 22 can be implemented by microphones and microphone drivers.

[0106] The receiving unit 14, the first image receiving unit 141, the first audio receiving unit 142, the first image transmitting unit 173, the first audio transmitting unit 174, the second transmitting unit 23, the second image receiving unit 24, and the second audio receiving unit 25 are usually implemented by wireless communication means, but may also be implemented by wired communication means. The wireless communication means is, for example, Wi-Fi, but is not limited to that.

[0107] The reception unit 15 and the operation reception unit 151 can be implemented using device drivers for input means such as touch panels and keyboards, or control software for menu screens.

[0108] The processing unit 16, speed acquisition unit 162, display control unit 163, acquisition determination unit 164, environment acquisition unit 165, state acquisition unit 166, state determination unit 167, image recognition unit 168, and placement determination unit 169 can typically be implemented using a processor, memory, etc. The processing procedures of the processing unit 16, etc., are typically implemented in software, and this software is recorded on a recording medium such as ROM. However, it may also be implemented in hardware (dedicated circuitry). The processor can be a CPU, MPU, GPU, etc., and the type is not limited.

[0109] The position acquisition unit 161 is implemented, for example, by a GPS receiver.

[0110] The output unit 17 and the warning unit 172 may or may not be considered to include output devices such as displays and speakers. The output unit 17 can be implemented using driver software for an output device, or driver software for an output device and an output device.

[0111] The display unit 171 and the second image display unit 26 may or may not be considered to include an output device such as a display. The output unit 17 can be implemented by driver software for an output device, or by driver software for an output device and an output device, etc.

[0112] The first audio output unit 175 and the second audio output unit 27 may or may not be considered to include an output device such as a speaker. The output unit 17 can be implemented using driver software for the output device, or a driver software for the output device and the output device itself.

[0113] Next, an example of the operation of the in-vehicle monitoring device 1 will be explained using the flowchart in Figure 4.

[0114] (Step S401) The receiving unit 14 determines whether or not it has received images, etc. from one or more camera devices 2. If images, etc. have been received, the unit proceeds to step S402; otherwise, it proceeds to step S411. Images, etc. can be images, or images and sound. Sound may also be referred to as sound information, audio information, or sound.

[0115] (Step S402) The speed acquisition unit 162 acquires the movement speed of the in-vehicle monitoring device 1. An example of this speed acquisition process will be explained using the flowchart in Figure 5. Note that the movement speed of the in-vehicle monitoring device 1 is usually the movement speed of the vehicle in which the in-vehicle monitoring device 1 is located.

[0116] (Step S403) The display control unit 163 determines whether the movement speed satisfies the image display conditions. If the display conditions are met, the process proceeds to step S404; otherwise, the process proceeds to step S409. Note that satisfying the display conditions is equivalent to not satisfying the non-display conditions. The display conditions are, for example, that the movement speed is less than or equal to a threshold.

[0117] (Step S404) The first audio output unit 175 outputs audio if the received image or other data contains audio. Such audio is the audio of the person being monitored. This audio may also be the sound from the cargo bed.

[0118] (Step S405) The environment acquisition unit 165 determines the brightness. An example of this brightness determination process will be explained using the flowchart in Figure 6.

[0119] (Step S406) The display unit 171 displays one or more images received in step S401 on a screen with brightness according to the brightness acquired in step S405. Here, if there are images of people in the back seat and images of luggage in the cargo area, it is preferable to display them in a way that makes them visually distinguishable. Displaying them in a way that makes them visually distinguishable can be done, for example, by changing the size of the window or by changing the display of the window frame (color of the frame lines, line type, thickness, etc.).

[0120] (Step S407) The state determination unit 167, etc., makes a determination and performs processing regarding the state of the object being monitored. An example of such state determination processing will be explained using the flowchart in Figure 7.

[0121] (Step S408) The processing unit 16 stores the image and other data received in step S401 in a buffer (not shown). The process returns to step S401.

[0122] (Step S409) The display control unit 163 determines whether or not an image (in this case, a camera image) is currently being displayed on the screen. If a camera image is being displayed, the process proceeds to step S410; otherwise, the process returns to step S401.

[0123] (Step S410) The display control unit 163 instructs the display unit 171 to hide the camera image displayed on the screen. The display unit 171 hides the camera image. Proceed to step S407.

[0124] (Step S411) The operation reception unit 151 determines whether or not it has received an operation request regarding the camera image displayed on the screen. If an operation request has been received, the process proceeds to step S412; otherwise, the process proceeds to step S413.

[0125] (Step S412) The display control unit 163 instructs the display unit 171 to change the camera image displayed according to the received operation. The display unit 171 changes the display of the camera image according to the operation. The process returns to step S401.

[0126] (Step S413) The abandonment decision unit 169, etc., performs the abandonment decision process for the caregiver. Return to step S401. An example of the abandonment decision process will be explained using the flowchart in Figure 8.

[0127] In the flowchart shown in Figure 4, processing is terminated by power-off or processing termination interrupts.

[0128] Next, an example of the speed acquisition process in step S402 will be explained using the flowchart in Figure 5.

[0129] (Step S501) The position acquisition unit 161 acquires the current position (P1).

[0130] (Step S502) The speed acquisition unit 162 acquires the current time (t1) from a clock (not shown).

[0131] (Step S503) The speed acquisition unit 162 stores the current position (P1) and current time (t1) as pairs in a buffer (not shown).

[0132] (Step S504) The speed acquisition unit 162 acquires the latest position (P0) and time (t0) in the buffer.

[0133] (Step S505) The speed acquisition unit 162 calculates the movement speed using the formula "Movement speed = |P1-P0| / (t1-t0)". It then returns to the higher-level processing unit. Note that |P1-P0| is the distance between position (P1) and position (P0).

[0134] Next, an example of the brightness determination process in step S405 will be explained using the flowchart in Figure 6.

[0135] (Step S601) The environment acquisition unit 165 determines whether the in-vehicle monitor device 1 can acquire illuminance. If external illuminance can be acquired, the process proceeds to step S602; otherwise, the process proceeds to step S604.

[0136] (Step S602) The environment acquisition unit 165 acquires the illuminance.

[0137] (Step S603) The display control unit 163 obtains the brightness corresponding to the illuminance. It returns to the higher-level processing. The display control unit 163, for example, refers to a correspondence table having two or more correspondence information items having illuminance ranges and brightness, and obtains the brightness that is paired with the illuminance range that includes the obtained illuminance from the correspondence table. For example, if the obtained illuminance is above a threshold, the display control unit 163 obtains the brightness corresponding to "bright mode", and if the obtained illuminance is below the threshold, it obtains the brightness corresponding to "dark mode".

[0138] (Step S604) The environment acquisition unit 165 acquires the latest current position acquired by the position acquisition unit 161.

[0139] (Step S605) The environment acquisition unit 165 acquires the time corresponding to the latest current location.

[0140] (Step S606) The environment acquisition unit 165 refers to map information (not shown) and determines the region from the latest current location.

[0141] (Step S607) The environment acquisition unit 165 acquires ambient brightness information (for example, sunrise time and sunset time) corresponding to the determined area from the ambient brightness management unit 111.

[0142] (Step S608) The environment acquisition unit 165 acquires brightness using the time and ambient brightness information acquired in step S605. It then returns to the higher-level processing.

[0143] Next, an example of the state determination process in step S407 will be explained using the flowchart in Figure 7.

[0144] (Step S701) The state acquisition unit 166 acquires one or more images.

[0145] (Step S702) The state acquisition unit 166 acquires the coordinate values ​​of two or more feature points from one or more images. The coordinate values ​​of feature points are, for example, the coordinate values ​​of the joints of a person and the center point of the face. The coordinate values ​​of feature points are, for example, the coordinate values ​​of two or four points that define a rectangle containing the outline of the face (a rectangle surrounding the face) and the coordinate values ​​of the eye positions. Note that this process is a known image processing method.

[0146] (Step S703) The state acquisition unit 166 acquires the degree of posture distortion from, for example, the coordinate values ​​of two or more feature points in each of the one or more images acquired in step S702. The state determination unit 167 determines whether the degree of posture distortion satisfies the attention alert condition (the degree of distortion is equal to or greater than a threshold). If the attention alert condition is met, the process proceeds to step S704; otherwise, the process proceeds to step S705.

[0147] The state acquisition unit 166 may, for example, acquire the similarity between the coordinate values ​​of two or more feature points of the latest image acquired in step S702 and the reference state information (in this case, the coordinate values ​​of two or more reference feature points), and the state determination unit 167 may determine whether the similarity satisfies the warning condition (the similarity is below or equal to a threshold).

[0148] (Step S704) The warning unit 172 outputs warning information. It returns to the higher-level processing. Preferably, the warning information here is a warning that the posture is compromised.

[0149] (Step S705) The state acquisition unit 166 acquires the degree of movement of the person being monitored (e.g., a child) using two or more images. The degree of movement is a value calculated by a decreasing function that uses the similarity of two images constituting the recorded video as a parameter.

[0150] (Step S706) The state determination unit 167 determines whether the degree of movement satisfies the warning condition (the degree of movement is equal to or greater than the threshold). If the warning condition is met, proceed to step S707; otherwise, proceed to step S708.

[0151] (Step S707) The warning unit 172 outputs warning information. It returns to the higher-level processing. Preferably, the warning information here is a warning that the movement is rapid.

[0152] (Step S708) The state acquisition unit 166 uses two temporally consecutive images, each lasting for a predetermined period of time or longer, to acquire the degree of inactivity, which is the degree to which the person being monitored has not moved for a predetermined period of time or longer. The degree of inactivity is calculated, for example, by an increasing function whose parameter is the time during which the similarity between the two images remains above a threshold.

[0153] (Step S709) The state determination unit 167 determines whether the degree of inactivity satisfies the warning condition (the degree of inactivity is equal to or greater than the threshold). If the warning condition is met, proceed to step S710; otherwise, proceed to step S711.

[0154] (Step S710) The warning unit 172 outputs warning information. It returns to the higher-level processing. Preferably, the warning information here is a warning that there is no movement.

[0155] (Step S711) The state acquisition unit 166 uses two temporally consecutive images, specifically images lasting for a predetermined duration or longer, to acquire the degree to which the face of the person being monitored is not facing forward. The degree of non-frontal view is calculated, for example, by an increasing function whose parameter is the continuous time during which the face of the person being monitored is not facing forward.

[0156] (Step S712) The state determination unit 167 determines whether the non-frontal degree satisfies the warning condition (the non-frontal degree is equal to or greater than the threshold). If the warning condition is met, proceed to step S713; otherwise, proceed to step S714.

[0157] (Step S713) The warning unit 172 outputs warning information. It returns to the higher-level processing. Preferably, the warning information here is a warning that the user has not been facing forward for a predetermined time or longer.

[0158] (Step S714) The status acquisition unit 166 acquires a reference status image. The reference status image is an image taken at the start of operation or at the start of shooting. For example, the reference status image is an image of the cargo bed taken at the time of departure, or an image of a child sitting securely in a child car seat at the time of departure.

[0159] (Step S715) The state determination unit 167 compares the image with a reference state image and determines whether or not the warning conditions are met. If the warning conditions are met, the process proceeds to step S716; otherwise, it returns to the higher-level processing.

[0160] (Step S716) The warning unit 172 outputs warning information. It returns to the higher-level processing. The warning information here is preferably, for example, warning information that something is wrong with the package.

[0161] In the flowchart in Figure 7, the process returned to the higher level after outputting one warning message, but it is also possible to output two or more warning messages.

[0162] Next, an example of the placement decision process in step S411 will be explained using the flowchart in Figure 8.

[0163] (Step S801) The placement determination unit 169 determines whether it is possible to communicate with one or more camera devices 2. If communication is possible, it returns to the higher-level processing; otherwise, it proceeds to step S802.

[0164] (Step S802) The placement determination unit 169 acquires the latest image from a buffer (not shown).

[0165] (Step S803) The presence / absence determination unit 169 determines whether or not a person is present in the latest image. If a person is present, the process proceeds to step S804; otherwise, the process returns to the higher-level processing.

[0166] (Step S804) The warning unit 172 outputs warning information. It returns to the higher-level processing. The warning information here indicates that a person has been left unattended in the vehicle.

[0167] The following describes specific operation examples of the in-vehicle monitoring system A in this embodiment. Five specific examples are described below. Specific example 1 is when the in-vehicle monitoring device 1 controls the display of the child being monitored by hiding the image when the travel speed exceeds a threshold (here, "20 km / h"). In other words, the display condition is "speed less than 20 km / h". Specific example 2 is when the in-vehicle monitoring device 1 automatically adjusts the screen brightness according to environmental information. Specific example 3 is when the in-vehicle monitoring device 1 outputs warning information when the posture or other condition of the child being monitored meets the warning conditions. Specific example 4 is when the in-vehicle monitoring device 1 performs a function to prevent the child being monitored from being left unattended in the car. Specific example 5 is when the in-vehicle monitoring device 1 controls the display of the displayed image based on the monitor's operation of the in-vehicle monitoring device 1.

[0168] Now, let's assume that user X, who is acting as a monitor, has installed an application on their smartphone that enables the functions of the in-car monitoring device 1 described above. Then, let's assume that user X has placed their smartphone in their car C as shown in Figure 1 and launched the application. Let's also assume that car C has, for example, camera devices 2a and 2b installed in the camera device 2 shown in Figure 1, and that when the engine of car C is started, the power to all camera devices 2 turns ON.

[0169] (Specific example 1) Assume that two children are sitting in the back seat of car C. When user X turns on the engine of car C and starts driving, images of each of the two children are transmitted wirelessly from camera devices 2a and 2b to the in-car monitor device 1.

[0170] The first image receiving unit 141 of the in-vehicle monitor device 1 receives images transmitted from camera devices 2a and 2b. The display unit 171 displays the images from each of the two camera devices (2a, 2b) in separate windows of a double-window system with the screen split into two, or using Picture-in-Picture (PiP).

[0171] As the driving of car C progresses, the speed acquisition unit 162 acquires a moving speed of "20 km / h" through the speed acquisition process explained using the flowchart in Figure 5. The display control unit 163 then determines that the display condition "speed less than 20 km / h" is not met. Next, the display control unit 163 hides the displayed image. It is preferable that the state determination process at S407 in the flowchart of Figure 4 and the process of accumulating the received image and sound (S408) continue even while the image is hidden.

[0172] Furthermore, when car C comes to a stop due to a red light, and the display control unit 163 determines that the speed of movement meets the display conditions, the display unit 171 displays the images transmitted from each of the two camera devices (2a, 2b) in each of the two double-windowed screens, or in Picture-in-Picture (PiP) mode.

[0173] As shown in Figure 9, buttons corresponding to multiple camera devices 2 are displayed, and the camera may be switched and images displayed according to the user X's instructions.

[0174] (Specific example 2) Currently, the external brightness management unit 111 of the in-vehicle monitoring device 1 stores the external brightness management table shown in Figure 10. This external brightness management table manages the sunrise and sunset times for each region and each day. In Figure 9, the "each day" refers to only the first day of each month, but it is also possible to manage the sunrise and sunset times for every day. The environment acquisition unit 165 here acquires brightness corresponding to "bright mode" from a threshold (e.g., "30 minutes") after sunrise until a threshold (e.g., "15 minutes") before sunset, and acquires brightness corresponding to "dark mode" during other time periods.

[0175] In the above situation, the in-vehicle monitor device 1 displays the images transmitted from the two camera devices (2a, 2b).

[0176] Then, when displaying the image, the environment acquisition unit 165 acquires brightness using the brightness determination process explained using the flowchart in Figure 6. In other words, the environment acquisition unit 165 determines whether or not the in-vehicle monitor device 1 can acquire illuminance. Here, we assume that the environment acquisition unit 165 has determined that the in-vehicle monitor device 1 cannot acquire illuminance. Next, the environment acquisition unit 165 acquires the latest current location (P) acquired by the location acquisition unit 161. The environment acquisition unit 165 also acquires the date (12 / 1) and time (7:30) from a clock (not shown). Next, the environment acquisition unit 165 refers to map information (not shown) and acquires the region "Tokyo" from the latest current location. Next, the environment acquisition unit 165 acquires ambient brightness information (901 in Figure 9) corresponding to the region "Tokyo". Next, the environment acquisition unit 165 acquires the brightness corresponding to "dark mode" because the acquired date (12 / 1) and time (7:30) are exactly the time of sunrise. Then, the display unit 171 displays the two received images on a screen with the brightness corresponding to "dark mode".

[0177] (Specific example 3) The state determination unit 167 of the in-vehicle monitoring device 1 outputs warning information when the child's posture meets the warning conditions, based on the state determination process explained using the flowchart in Figure 7.

[0178] In other words, the state acquisition unit 166 analyzes an image of the child at the time when user X starts driving car C, obtains the coordinate values ​​of two or more feature points in the image of the child, and stores the set of coordinate values ​​of these two or more feature points as reference state information in the storage unit 11.

[0179] Then, while user X drives car C, the state acquisition unit 166 analyzes the image of the child and acquires a set of coordinate values, which are the coordinate values ​​of two or more feature points in the image of the child (1102 in Figure 11). The state acquisition unit 166 then acquires the similarity between the reference state information and the acquired set of coordinate values ​​(for example, the reciprocal of the degree of posture distortion). Next, the state acquisition unit 166 determines that the similarity satisfies the attention condition (here, "the similarity is below or equal to a threshold"). The similarity of the two sets of coordinate values ​​may be obtained, for example, by the sum of the distances between two corresponding points in each of the two sets of coordinate values, or by an increasing function with that sum as a parameter. The similarity of the two sets of coordinate values ​​may be obtained, for example, by a decreasing function with the distance between two vectors whose elements are the coordinate values ​​in each of the two sets of coordinate values ​​as a parameter. The method for calculating the similarity of the two sets of coordinate values ​​is not specified.

[0180] Next, the warning unit 172 outputs warning information indicating that the child in the back seat is in an unbalanced posture. Here, the warning unit 172 may output an audio message saying, "The child in the back seat is in an unbalanced posture," or it may display warning information, or it may emit light, etc.

[0181] Furthermore, for example, even if the display conditions are not met and the child's image is not displayed (while driving at high speed), it is preferable for the warning unit 172 to output warning information.

[0182] (Specific example 4) Let's assume that the first image receiving unit 141 of the in-car monitor device 1 is receiving an image transmitted from the camera device 2a. Let's also assume that the state determination unit 167 is recognizing a person in the received image in order to acquire state information (degree of posture distortion, degree of movement, etc.). In this specific example, let's assume that only the camera device 2a, which is filming one child, is powered on.

[0183] In this situation, the departure determination unit 169 determines whether or not it is possible to communicate with the camera device 2a. The departure determination unit 169 determines that it is not possible to communicate with the camera device 2a. In other words, user X leaves the vehicle with the in-car monitor device 1, which is a portable terminal.

[0184] Next, the warning unit 172 outputs warning information indicating that a person has been left behind in car X.

[0185] (Specific example 5) Assume that the screen of the in-car monitor device 1 is a touch panel. Now, an image of a child is displayed across the entire screen of the in-car monitor device 1 (Figure 12, 1201). In 1201, only the image of the child is displayed on the screen.

[0186] Then, suppose the user touches the screen, for example, when an image of a child is displayed across the entire screen of the in-car monitor device 1. The display control unit 163 then instructs the display unit 171 to display the operation interface (in this case, buttons) from hidden. Next, the display unit 171 displays the operation interface on the screen.

[0187] Furthermore, let's assume that currently, an image with a small zoom level is displayed, as shown in screen 1301 of Figure 13. Screen 1301 displays a zoom button 1302 and a zoom out button 1303. Now, let's assume that user X presses the zoom button 1302. The operation reception unit 151 then receives the zoom command and enlarges the displayed image (1304 in Figure 13).

[0188] Furthermore, if user X drags the center point shape (+) (1401 in Figure 14) displayed on the screen, the display unit 171 changes the area displayed within the image. It is preferable that the point indicated during dragging can be any point on the screen.

[0189] As described above, according to this embodiment, the driver and passenger can appropriately view images of the person being monitored inside the vehicle, depending on the vehicle's speed.

[0190] Furthermore, according to this embodiment, depending on the environment, the driver, passenger, etc., can view the captured image of the person being monitored inside the vehicle with appropriate brightness.

[0191] Furthermore, according to this embodiment, even when illuminance cannot be obtained, the driver, passenger, etc., can view the captured image at an appropriate brightness according to the ambient light, allowing them to see the object being monitored inside the vehicle.

[0192] Furthermore, according to this embodiment, by analyzing images of the person being monitored inside the vehicle, it is possible to alert the driver, passenger, etc., according to the acquired state of the person being monitored.

[0193] Furthermore, according to this embodiment, by analyzing images of the person being monitored inside the vehicle and comparing the acquired state of the monitored person with a reference state, it is possible to alert the driver, passenger, etc.

[0194] Furthermore, according to this embodiment, it is possible to effectively prevent the person being protected from being left unattended inside the vehicle.

[0195] Furthermore, according to this embodiment, the screen can be operated to appropriately view the status of the protected object.

[0196] Furthermore, according to this embodiment, by providing an environment in which the protected person can see images of the driver and the person in the passenger seat, the protected person can be made to feel safe even while in transit.

[0197] The processing in this embodiment may be implemented by software. This software may be distributed by software download or the like. Alternatively, this software may be recorded on a recording medium such as a CD-ROM and distributed. This also applies to other embodiments in this specification. The application software that implements the in-vehicle monitor device 1 in this embodiment is the following program. In other words, this program causes a computer to function as a receiving unit that receives images of the interior of the vehicle transmitted by one or more camera devices that photograph the rear seat or cargo area of ​​the vehicle, a display unit that displays the images received by the receiving unit, a position acquisition unit that acquires the current position at two or more different points in time, a speed acquisition unit that uses the two or more current positions acquired by the position acquisition unit to acquire the speed of movement, and a display control unit that changes the display of the images according to the speed of movement. It is preferable that this computer be a mobile terminal such as a smartphone.

[0198] Figure 15 is a block diagram of a computer system 300 that executes the program described herein to realize the in-vehicle monitoring device 1 and the like according to the various embodiments described above.

[0199] In Figure 15, the computer system 300 includes a computer 301 with a CD-ROM drive, a keyboard 302, a mouse 303, a monitor 304, a camera 305, and a microphone 306.

[0200] In Figure 15, the computer 301 includes, in addition to the CD-ROM drive 3012, an MPU 3013, a bus 3014 connected to the CD-ROM drive 3012, a ROM 3015 for storing programs such as boot-up programs, a RAM 3016 connected to the MPU 3013 for temporarily storing application program instructions and providing temporary storage space, and a hard disk 3017 for storing application programs, system programs, and data. Although not shown here, the computer 301 may further include a network card for providing connectivity to a LAN.

[0201] The program that causes the computer system 300 to execute the functions of the in-vehicle monitoring device 1, etc., as described above, may be stored on CD-ROM 3101, inserted into CD-ROM drive 3012, and then transferred to hard disk 3017. Alternatively, the program may be transmitted to computer 301 via a network (not shown) and stored on hard disk 3017. The program is loaded into RAM 3016 during execution. The program may also be loaded directly from CD-ROM 3101 or the network.

[0202] The program does not necessarily have to include an operating system (OS) or third-party program that causes the computer 301 to execute functions such as the in-vehicle monitoring device 1 of the above-described embodiment. The program only needs to include the instruction portion that calls appropriate functions (modules) in a controlled manner and obtains the desired result. How the computer system 300 operates is well known, so a detailed explanation is omitted.

[0203] In the above program, steps such as sending information and receiving information do not include hardware-based processing, such as processing performed by a modem or interface card in the transmission step (processing that can only be performed by hardware).

[0204] Furthermore, the computer running the above program may be a single computer or multiple computers. In other words, it may perform centralized processing or distributed processing.

[0205] Furthermore, it goes without saying that in each of the above embodiments, two or more communication means present in a single device may be physically implemented in a single medium.

[0206] Furthermore, in each of the above embodiments, each process may be implemented by centralized processing by a single device, or by distributed processing by multiple devices.

[0207] It goes without saying that the present invention is not limited to the embodiments described above, and various modifications are possible, all of which are also included within the scope of the present invention. [Industrial applicability]

[0208] As described above, the in-vehicle monitoring device according to the present invention has the effect of allowing the driver, passenger, etc., to appropriately view images of the person being monitored inside the vehicle, and is therefore useful as an in-vehicle monitoring device. [Explanation of symbols]

[0209] A In-car monitoring system 1. In-vehicle monitoring device 2 Camera device 11 Storage Unit 12 First Photography Department 13 First microphone 14 Receiving Unit 15 Reception Department 16 Processing Unit 17 Output section 21 Second Photography Department 22 Second microphone 23 Second Transmission Section 24 Second image receiving unit 25 Second audio receiver 26 Second image display section 27 Second audio output section 111 External Brightness Management Department 112 Camera Management Department 141 First image receiving unit 142 First audio receiver 151 Operation Reception Section 161 Position acquisition part 162 Speed ​​acquisition section 163 Display Control Unit 164 Acquisition Judgment Department 165 Environmental Acquisition Department 166 Status acquisition unit 167 State determination unit 168 Image recognition unit 169 Placement Decision Unit 171 Display section 172 Alert section 173 First Image Transmission Unit 174 First Voice Transmitter 175 First audio output section

Claims

1. Computers, A receiving unit that receives images of the interior of a vehicle transmitted by one or more camera devices that photograph the rear seat or cargo area of ​​a vehicle, The receiving unit includes a display unit that displays the image received by the receiving unit, A position acquisition unit that acquires the current position at two or more different points in time, The position acquisition unit has a speed acquisition unit that acquires the movement speed using the two or more acquired current positions, A program for causing a display control unit to function as a display control unit that changes the display of the image according to the aforementioned movement speed.

2. The aforementioned computer, It will further function as an environmental information acquisition unit to acquire environmental information. The display control unit, The program according to claim 1 for causing the computer to function in such a way as to change the brightness of the screen on which the image is displayed in accordance with the environmental information.

3. It is possible to access the ambient brightness management unit, which stores ambient brightness information that identifies the brightness of the outside, in association with two or more location pieces. The aforementioned computer, The aforementioned environmental acquisition unit is further configured to function as an acquisition determination unit that determines whether or not it can acquire illuminance. The aforementioned environment acquisition unit, If illuminance can be obtained, it will be obtained; otherwise, it will obtain the time and location information corresponding to the current location. The display control unit, The program according to claim 2 for causing the computer to function such that, if the environment acquisition unit acquires illuminance, it changes the brightness of the screen on which the image is displayed according to the illuminance, and if the environment acquisition unit cannot acquire illuminance, it acquires the ambient brightness information associated with the location information corresponding to the time and the current location, and changes the brightness of the screen on which the image is displayed according to the ambient brightness information.

4. The aforementioned computer, A state acquisition unit analyzes the image received by the receiving unit and acquires state information to identify the state of the subject being photographed in the rear seat or the cargo area. A state determination unit that determines whether the state information acquired by the state acquisition unit satisfies the warning conditions, The program according to claim 1, which further functions as a warning unit that outputs warning information when the state determination unit determines that the warning conditions are met.

5. The aforementioned state determination unit, The program according to claim 4 for causing the computer to function by comparing the state information acquired by the state acquisition unit with reference state information to determine whether or not the warning conditions are met.

6. The aforementioned computer, The receiving unit includes an image recognition unit that recognizes a person in the image received by the receiving unit, The abandoned determination unit determines whether the abandoned condition is met, which is that communication with the camera device is lost when the image recognition unit recognizes the person, The program according to claim 1, which further functions as a warning unit that outputs warning information when the abandonment determination unit determines that the abandonment conditions are met.

7. The aforementioned computer, The display unit is further configured to function as an operation reception unit that receives operations on the image displayed by the display unit. The display control unit, The program according to claim 1 for causing the computer to function in such a way as to change the display of the image in response to the aforementioned operation.

8. The aforementioned computer, The first shooting unit acquires the first image of the driver or passenger, The program according to claim 1, further functioning as a first image transmission unit that transmits the first image to the camera device.

9. The aforementioned computer, The first microphone collects the primary sound, A first audio transmission unit that transmits the first audio to the camera device, A first audio receiving unit that receives a second audio signal from the camera device, This unit further functions as a first audio output unit that outputs the second audio signal. The camera device is A second microphone to collect the second audio signal, A second transmitting unit that transmits the second audio, The in-vehicle monitoring device includes a second receiving unit that receives the first audio signal transmitted, The program according to claim 8, further comprising a second audio output unit that outputs the first audio.

10. The receiving unit is Images are received from two or more camera devices. The aforementioned display unit is The program according to claim 1, which causes the computer to function as a device that displays two or more images received from two or more camera devices.

11. It is possible to access the camera management unit that stores camera identifiers indicating which cameras are hidden or displayed. The display control unit, The program according to claim 10, which causes the computer to function using the camera identifier to display only the image corresponding to the camera device to be displayed among the two or more camera devices.

12. At least one of the two camera devices is photographing the person being protected, and the other is photographing the luggage. The display control unit, The program according to claim 10, which causes the computer to function as a device that instructs the display unit to display the image of the protected person and the image of the luggage in a distinguishable manner.

13. An in-vehicle monitoring method comprising all steps performed by the program according to any one of claims 1 to 12.

14. An in-vehicle monitoring device capable of performing all steps of the program described in any one of claims 1 to 12.

Images