Notification device, notification method, and program

Abstract

The present invention includes: an acquisition unit (52) for acquiring a photographed image obtained by photographing the wheels of a vehicle (200); a receiving unit (51) for receiving a designation of a reference position; a determination unit (53) for determining, based on the photographed image, a current wheel nut at the same position as a reference wheel nut at the reference position; a determination unit (53) for determining, based on the reference position, a wheel nut at the same position as a reference wheel nut included in a plurality of wheel nuts in a photographed image previously acquired by the acquisition unit (52), i.e., a reference wheel nut, i.e., a reference wheel nut, i.e., a reference wheel nut, at the same position as the reference wheel nut at the reference position; a detection unit (54) for detecting looseness of the current wheel nut by detecting the rotation of the current wheel nut determined by the determination unit (53) relative to the reference wheel nut included in the reference image; and a notification unit (56) for notifying the user that the current wheel nut is loose when the detection unit (54) detects that the current wheel nut is loose.

Description

Technical Field

[0001] This invention relates to a notification device, notification method, and procedure for notifying that wheel nuts are loose. Background Technology

[0002] Technology for automatically detecting loose wheel nuts is known (for example, see Patent Document 1). Patent Document 1 describes a method for detecting loose wheel nuts by measuring the strain of a spacer-shaped component subjected to the tightening force of the wheel nut using a strain sensor.

[0003] Existing technical documents

[0004] Patent documents

[0005] Patent Document 1: Japanese Patent Application Publication No. 2016-187977 Summary of the Invention

[0006] The technical problem that the invention aims to solve

[0007] In the method described in Patent Document 1, the operation for detecting loose wheel nuts is complicated because strain sensors need to be installed on each wheel nut, and power and communication devices for the strain sensors also need to be ensured.

[0008] The present invention was made in view of this purpose, and its object is to provide a notification device, notification method and procedure that can simplify the operation of detecting loose wheel nuts.

[0009] means for solving problems

[0010] The notification device of the first aspect of the present invention includes: an acquisition unit for acquiring a photographed image of a vehicle's wheel; a receiving unit for receiving a designation of a reference position, the reference position being used to identify the position of a plurality of current wheel nuts provided on the wheel in the photographed image; a storage unit for storing, in association, the photographed image previously acquired by the acquisition unit, i.e., a reference image, and the reference position of the reference image; a determination unit for selecting one of the plurality of wheel nuts included in the reference image, i.e., a reference wheel nut, based on the reference position of the reference image, and determining, for the reference position received by the receiving unit, a current wheel nut at the same position as the reference wheel nut according to the photographed image acquired by the acquisition unit; a detection unit for detecting loosening of the current wheel nut by detecting whether the current wheel nut determined by the determination unit rotates relative to the reference wheel nut included in the reference image; and a notification unit for notifying a user that the current wheel nut is loose when the detection unit detects that the current wheel nut is loose.

[0011] The notification device may further include: a display control unit that causes the display unit to display the previously captured image, as instructed by the user, as a preview image; the receiving unit may also accept a designation of a reference position for the display area of ​​the preview image. The acquiring unit may also acquire the captured image when the receiving unit receives the designation of the reference position.

[0012] The notification device may further include a display control unit that displays a previously captured image, as instructed by the user, as a pre-display image, and displays the display image including a frame for specifying the reference position. If the detection unit detects that more than a predetermined number of wheels included in the captured image are loose, the notification unit may also notify the user that the wheels included in the reference image are different from those included in the captured image. The detection unit may also calculate the rotation angle of the current wheel nut relative to the reference wheel nut, and detect the loosening of the current wheel nut if the calculated rotation angle is above a threshold.

[0013] The notification method of the second aspect of the present invention includes the following steps performed by a computer: acquiring a photographic image of a vehicle's wheel; accepting a designation of a reference position for identifying the position of a plurality of current wheel nuts on the wheel in the photographic image; storing the previously acquired photographic image, i.e., a reference image, in association with the reference position of the reference image; selecting one of the plurality of wheel nuts included in the reference image, i.e., a reference wheel nut, based on the reference position of the reference image, and determining, for the accepted reference position, a current wheel nut at the same position as the reference wheel nut according to the acquired photographic image; detecting loosening of the current wheel nut by detecting rotation of the determined current wheel nut relative to the reference wheel nut included in the reference image; and notifying the user that the current wheel nut is loose when loosening is detected.

[0014] The third-party program of the present invention enables a computer to perform the following steps: acquiring a photographic image of a vehicle's wheel; accepting a designation of a reference position for identifying the positions of multiple current wheel nuts on the wheel in the photographic image; storing the previously acquired photographic image, i.e., a reference image, in association with the reference position of the reference image; selecting one of the multiple wheel nuts included in the reference image, i.e., a reference wheel nut, based on the reference position of the reference image, and determining, for the accepted reference position, a current wheel nut at the same position as the reference wheel nut, according to the acquired photographic image; detecting loosening of the current wheel nut by detecting rotation of the determined current wheel nut relative to the reference wheel nut included in the reference image; and notifying the user of the current wheel nut's loosening when loosening is detected.

[0015] The effects of the invention

[0016] According to the present invention, the operation of detecting loose wheel nuts is simplified. Attached Figure Description

[0017] Figure 1 This is a diagram showing an outline of the notification device according to an embodiment.

[0018] Figure 2 This is a diagram showing the structure of the notification device.

[0019] Figure 3 This is a diagram illustrating an example of the process of receiving the reference position.

[0020] Figure 4 This is a diagram illustrating an example of a detection method for loose wheel nuts performed by the detection department.

[0021] Figure 5 This diagram illustrates an example of a method by which a notification department notifies the user of a loose wheel nut.

[0022] Figure 6 This is a diagram showing the processing sequence of wheel nut loosening detection performed by the notification device.

[0023] Figure 7 This is a diagram illustrating another example of the receiving unit accepting the specified processing of the reference position. Detailed Implementation

[0024] [Overview of the notification device]

[0025] Figure 1This is a diagram showing an outline of the notification device 100 according to this embodiment. The notification device 100 is a device managed by the operator of the vehicle 200, such as an information terminal like a smartphone or tablet. The vehicle 200 is, for example, a commercial vehicle such as a truck. The notification device 100 detects the loosening of multiple wheel nuts provided on the wheels 20 of the vehicle 200.

[0026] First, the notification device 100 acquires a photographed image of the wheel 20. The notification device 100 reads a reference image from its storage unit. This reference image is a photograph of the same wheel 20 as the acquired image, taken previously. For example, the reference image may be one acquired when it was previously confirmed that the wheel nuts of the wheel 20 were not loose. Alternatively, the reference image may be one acquired during the last photograph of the wheel 20. The notification device 100 then identifies a current wheel nut from among the multiple wheel nuts included in the photographed image (hereinafter referred to as the current wheel nut) that is in the same position as one of the wheel nuts included in the reference image (hereinafter also referred to as the reference wheel nut).

[0027] Details will be described later. The notification device 100 detects the looseness of the current wheel nut by determining whether the current wheel nut is rotated relative to a reference wheel nut included in the reference image. When the notification device 100 detects that the current wheel nut is loose, it notifies the user that the current wheel nut is loose. In this way, the notification device 100 simplifies the operation of detecting the looseness of the wheel nut.

[0028] [Structure of the notification device]

[0029] Figure 2 This diagram illustrates the structure of the notification device 100. The notification device 100 includes a touch panel 1, a camera unit 2, a display unit 3, a storage unit 4, and a control unit 5. The touch panel 1 accepts user operations on the display surface of the display unit 3. The camera unit 2 captures images of the wheels 20 of the vehicle 200. The display unit 3 is a display that shows images, characters, etc. The storage unit 4 is composed of, for example, ROM (Read Only Memory) and RAM (Random Access Memory).

[0030] The control unit 5 is, for example, a CPU (Central Processing Unit). The control unit 5 functions as a receiving unit 51, an acquiring unit 52, a determining unit 53, a detecting unit 54, a display control unit 55, and a notification unit 56 by executing programs stored in the storage unit 4.

[0031] The receiving unit 51 accepts user operations on the touch panel 1. For example, the receiving unit 51 accepts operations at a specified reference position, which is used to identify the position of multiple current wheel nuts set on the wheel 20 in the captured image.

[0032] [Specification of reference position]

[0033] Figure 3 This is a diagram illustrating an example of the processing of the receiving unit 51 receiving a reference position. Figure 3 The image shows the situation when the camera unit 2 is taking pictures of the wheel 20. The previously captured image, which was taken at the user's instruction, is displayed on the display unit 3 as a preview image.

[0034] For example, suppose that the application of the notification device 100 for performing pre-operation checks on vehicle 200 is activated, and the user selects the wheel nut check from among the multiple pre-operation check items. During the wheel nut check, after a message instructing the display unit 3 to photograph the left front wheel 20 of vehicle 200 is displayed, when the receiving unit 51 receives an instruction from the user to display a pre-display image, the display unit... Figure 3 The pre-display image.

[0035] exist Figure 3 In this example, the pre-displayed image includes the wheels 20 of vehicle 200. Wheel 20 includes multiple current wheel nuts and an air valve 201. The receiving unit 51 accepts a designation of a reference position for the wheel 20 in the display area 60 of the pre-displayed image. In this example, the reference position is the position of the air valve 201.

[0036] The acquisition unit 52 acquires images of the wheels 20 of the vehicle 200 captured by the imaging unit 2. For example, when the receiving unit 51 receives a designation for a reference position, the acquisition unit 52 acquires the image. Figure 3 In the example, the display unit 3 displays a message: "Take a picture when the position of the air valve on the left front wheel of the vehicle is specified by touch operation." When the receiving unit 51 receives an operation specifying the position of the air valve as the reference position, the acquisition unit 52 acquires a picture of the wheel 20. Therefore, the user does not need to separately instruct the acquisition of the picture.

[0037] The determining unit 53 determines the relative positions of multiple current wheel nuts included in the captured image acquired by the acquiring unit 52 relative to a reference position. For example, based on the captured image acquired by the acquiring unit 52, the determining unit 53 determines the relative positions of the multiple current wheel nuts in the wheel 20, starting from the reference position, in a clockwise direction.

[0038] In the storage unit 4, the captured image of the wheel 20 previously acquired by the acquisition unit 52, i.e., the reference image, information indicating the reference position of the reference image, and vehicle identification information for identifying the vehicle 200 are stored in association. The reference position of the reference image is the position designated by the receiving unit 51 while the reference image is displayed on the display unit 3 as a pre-display image. The acquisition unit 52 associates the information indicating the reference position designated by the receiving unit 51 with the captured image, and stores it in the storage unit 4 as the reference image.

[0039] The determination unit 53 reads from the storage unit 4 a reference image corresponding to the wheel that corresponds to the wheel in the captured image, and information indicating the reference position of the reference image, from the reference images associated with the vehicle identification information selected by the user in the application of the notification device 100 for performing pre-operation checks of the vehicle 200. Multiple reference images corresponding to multiple wheels 20 of the vehicle 200 are associated with one vehicle identification information. As an example, assume that the determination unit 53 reads from the multiple reference images associated with the vehicle identification information selected by the user the message displayed on the display unit 3: "Captured when the position of the air valve on the left front wheel of the vehicle was specified via touch operation." Figure 3 The reference image of wheel 20, which includes the left front wheel, is shown.

[0040] The determining unit 53 selects one wheel nut from a plurality of current wheel nuts included in the read reference image as the reference wheel nut. The determining unit 53 selects the reference wheel nut based on the reference position of the reference image. For example, the determining unit 53 selects the first wheel nut clockwise from the reference position as the reference wheel nut. Based on the determined relative position of the current wheel nut, the determining unit 53 determines a current wheel nut 202 at the same position as the selected reference wheel nut relative to the reference position.

[0041] [Current wheel nut looseness detection]

[0042] The detection unit 54 detects the looseness of the current wheel nut by detecting the rotation of the current wheel nut relative to the reference wheel nut included in the reference image, as determined by the determination unit 53.

[0043] Figure 4This diagram illustrates an example of a method for detecting the looseness of the current wheel nut 202 performed by the detection unit 54. The detection unit 54 specifies a line segment L1 connecting the center of the current wheel nut 202, determined by the determination unit 53, to the axle of the wheel 20. The detection unit 54 selects the vertex among the six vertices of the hexagon of the current wheel nut 202 that has the smallest distance to the axle of the wheel 20. The detection unit 54 determines a line segment L2 connecting the selected vertex to the axle of the wheel 20. The detection unit 54 calculates the angle A formed by line segments L1 and L2. The detection unit 54 reads the angle A', which was also calculated in the reference image, from the storage unit 4.

[0044] When angle A' and angle A are inconsistent, it can be determined that the current wheel nut 202 has rotated since the reference image was captured. If the difference in the calculated angles is greater than or equal to a first threshold, the detection unit 54 detects that the current wheel nut 202 is loose. The first threshold is, for example, a preset angle that is the maximum allowable angle for changes in the angle of the reference wheel nut. The first threshold can also be the minimum angle that the detection unit 54 can detect. On the other hand, if the difference in the calculated angles is less than the first threshold, the detection unit 54 does not detect that the current wheel nut 202 is loose.

[0045] Furthermore, if the difference between the rotation angle of the reference wheel nut at the reference position relative to the reference image and the rotation angle of the current wheel nut 202 at the reference position relative to the image in which the current wheel nut 202 is captured is a second threshold or higher, the detection unit 54 may also detect that the current wheel nut 202 is rotating relative to the reference wheel nut.

[0046] For example, the detection unit 54 rotates the reference image so that the reference position is directly below the reference wheel nut. Similarly, the detection unit 54 rotates the captured image so that the reference position is directly below the current wheel nut 202 determined by the determination unit 53. The detection unit 54 compares the rotated reference wheel nut of the reference image with the rotated current wheel nut 202 of the captured image. The detection unit 54 determines the rotation angle of the current wheel nut 202 relative to the reference wheel nut. If the determined rotation angle is greater than or equal to a second threshold, the detection unit 54 detects that the current wheel nut 202 is loose. The second threshold is, for example, a preset angle that is the maximum value of the angle change relative to the reference wheel nut that is allowed. The second threshold may also be the minimum value of the angle that the detection unit 54 can detect.

[0047] The display control unit 55 causes the display unit 3 to display various images and characters. For example, when checking wheel nuts, the display control unit 55 causes the display unit 3 to display a message instructing the camera to photograph the left front wheel 20 of the vehicle 200. Figure 3As shown, the display control unit 55 displays a previously captured image, as instructed by the user, on the display unit as a preview image.

[0048] [Notification of loose wheel nuts]

[0049] When the detection unit 54 detects that the current wheel nut 202 is loose, the notification unit 56 notifies the user that the current wheel nut 202 is loose. Figure 5 This diagram illustrates an example of a method by which notification unit 56 notifies the current wheel nut 202 of looseness. Notification unit 56 displays a message indicating that looseness of the current wheel nut 202 has been detected. Notification unit 56 displays information indicating that detection unit 54 has detected a loose current wheel nut 202. Figure 5 In the example, the notification unit 56 attaches a double-circle mark indicating that the current wheel nut 202 is loose, detected by the detection unit 54, to the captured image of the wheel 20 and displays it on the display unit 3.

[0050] If the detection unit 54 detects that multiple current wheel nuts 202 included in the same wheel 20 are loose, there is a possibility that the wheel 20 included in the reference image is different from the wheel 20 included in the captured image. If the detection unit 54 detects that more than a predetermined number of current wheel nuts 202 included in the captured image are loose, the notification unit 56 notifies the user that the wheel included in the reference image is different from the wheel included in the captured image. The predetermined number is, for example, approximately half of the total number of current wheel nuts 202 included in the wheel 20.

[0051] [Procedure for detecting loose wheel nuts]

[0052] Figure 6 This diagram illustrates the processing sequence of wheel nut loosening detection performed by the notification device 100. This processing sequence, for example, begins when the receiving unit 51 receives an instruction from the user to display a pre-displayed image after a message instructing the display unit 3 to take a picture of any one of the multiple wheels 20 of the vehicle 200.

[0053] First, the display control unit 55 causes the display unit 3 to display a pre-display image (S101). The receiving unit 51 determines whether to accept the designation of a reference position (S102). If the receiving unit 51 accepts the designation of a reference position (S102 "Yes"), the acquisition unit 52 acquires a photographic image of the wheel 20 of the photographic vehicle 200 (S103). The determining unit 53 reads a reference image from the storage unit 4, which is a photographic image of the same wheel 20 previously acquired by the acquisition unit 52. The determining unit 53 selects an unselected wheel nut from a plurality of wheel nuts included in the reference image as the reference wheel nut (S104).

[0054] The determining unit 53 determines the current wheel nut 202 at the same position as the selected reference wheel nut (S105). The detection unit 54 detects whether the current wheel nut 202 is loose by detecting the rotation of the current wheel nut 202 determined by the determining unit 53 relative to the reference wheel nut. The notification unit 56 determines whether the detection unit 54 has detected the current wheel nut 202 being loose (S106). When the notification unit 56 determines that the detection unit 54 has detected the current wheel nut 202 being loose (S106 "Yes"), the notification unit 56 notifies the user that the current wheel nut 202 is loose (S107). The determining unit 53 determines whether there is an unselected wheel nut among the multiple wheel nuts included in the reference image (S108). If the determining unit 53 determines that there is no unselected wheel nut (S108 "No"), the process ends.

[0055] If the receiving unit 51 does not receive a designation for the reference position in the determination of S102 (S102 "No"), the determination of S102 is repeated. If the notification unit 56 determines in the determination of S106 that the detection unit 54 has not detected the current wheel nut 202 being loose (S106 "No"), the process proceeds to the determination of S108. If the determining unit 53 determines in the determination of S108 that there is an unselected wheel nut (S108 "Yes"), the process returns to S104.

[0056] [Effects of the Invention]

[0057] According to this embodiment, the detection unit 54 can detect the loosening of the current wheel nut 202 by detecting the rotation of the current wheel nut 202 relative to the reference wheel nut. Therefore, the notification device 100 can simplify the operation of detecting the loosening of the wheel nut.

[0058] [Specified variation of the reference position]

[0059] exist Figure 3 In the example, the receiving unit 51 accepts the designation of a reference position by using the location where the user's touch operation occurred as the reference position. However, the present invention is not limited to this. The receiving unit 51 may also use a frame for designating a reference position to accept the designation of a reference position. Figure 7 This diagram illustrates another example of the processing by which the receiving unit 51 accepts the designation of a reference position. Figure 7 In the example, the display control unit 55 causes the display unit 3 to display a pre-display image including a frame 61 for specifying a reference position. Figure 7In the example, the message "Please take a picture with the air valve of the vehicle's left front wheel inside the frame." is displayed. When the receiving unit 51 receives the user's operation on the shooting button 62, the acquisition unit 52 acquires the captured image of the wheel 20. At this time, the receiving unit 51 can also accept the operation of the shooting button 62 when the position within the frame 61 is designated as the reference position.

[0060] Furthermore, the receiving unit 51 is not limited to accepting a designation of a reference position for the display area 60 of the pre-displayed image. For example, the display control unit 55 displays a captured image of the wheel 20 acquired by the acquisition unit 52 on the display unit 3. The receiving unit 51 may also accept a designation of a reference position for the display area of ​​the captured image displayed on the display unit 3.

[0061] The present invention has been described above using embodiments, but the technical scope of the present invention is not limited to the scope described in the above embodiments, and various modifications and alterations can be made within its scope. For example, all or part of the device can be configured in any unit, functionally or physically distributed / integrated. Furthermore, new embodiments resulting from any combination of multiple embodiments are also included in the embodiments of the present invention. The effects of new embodiments resulting from combinations are the same as the effects of the original embodiments.

[0062] Explanation of reference numerals in the attached figures

[0063] 1. Touch panel

[0064] 2. Filming Department

[0065] 3 Display Section

[0066] 4. Storage Section

[0067] 5. Control Department

[0068] 20 Last Wheel

[0069] 20 wheels

[0070] 51 Receiving Department

[0071] 52 Acquisition Department

[0072] 53 Determining Department

[0073] 54 Testing Department

[0074] 55 Display Control Unit

[0075] 56 Notification Department

[0076] 60 display area

[0077] 61 frames

[0078] 62. Camera button

[0079] 100 Notification Device

[0080] 200 vehicles

[0081] 201 Air Valve

[0082] 202 Current wheel nut.

Claims

1. A notification device, characterized in that, include: The acquisition unit acquires images of the vehicle's wheels by taking photographs. The receiving unit accepts the designation of a reference position, which is used to identify the position of a plurality of current wheel nuts provided on the wheel in the captured image; The storage unit stores the captured image (i.e., the reference image) acquired by the acquisition unit in the past, and the reference position of the reference image in association; The determining unit selects one of a plurality of wheel nuts included in the reference image, namely the reference wheel nut, based on the reference position of the reference image. For the reference position received by the receiving unit, the current wheel nut at the same position as the reference wheel nut is determined according to the captured image obtained by the acquiring unit. The detection unit detects the looseness of the current wheel nut by detecting the rotation of the current wheel nut relative to the reference wheel nut included in the reference image, as determined by the determination unit. The notification unit notifies the user that the current wheel nut is loose when the detection unit detects that the current wheel nut is loose. as well as The display control unit displays a pre-display image and text. The pre-display image is an image obtained by overlaying a frame indicating the reference position onto a pre-capture image acquired in preparation for capturing the image before the user instructs the camera to take a picture. The text instructs that the part of the wheel that should be used as the reference position should be placed within the frame. When the receiving unit receives information instructing it to acquire the captured image, it accepts the position within the frame as the reference position.

2. The notification device according to claim 1, characterized in that, The acquisition unit acquires the captured image when the receiving unit receives the designation of the reference position.

3. The notification device according to claim 1 or 2, characterized in that, If the detection unit detects that more than a predetermined number of the current wheel nuts included in the captured image are loose, the notification unit notifies the user that the wheels included in the reference image are different from the wheels included in the captured image.

4. The notification device according to claim 1 or 2, characterized in that, The detection unit calculates the rotation angle of the current wheel nut relative to the reference wheel nut, and if the calculated rotation angle is above a threshold, it detects that the current wheel nut is loose.

5. A notification method, characterized in that, This includes the following steps that cause the computer to perform: The display unit shows a pre-capture image that is acquired in preparation for capturing images of the vehicle's wheels at a moment before the user instructs the user to take a picture, a pre-display image that overlaps a frame for specifying the position of a reference position for identifying the position of multiple current wheel nuts set on the wheel, and text indicating the part of the wheel that should be used as the reference position to be placed in the frame. The step of acquiring the captured image based on the received instruction to acquire the captured image; The step of accepting the position within the frame as the reference position in the captured image; The step of storing the previously acquired captured images, i.e., the reference images, and the reference positions of the reference images in association; Based on the reference position of the reference image, the step of selecting one of the multiple wheel nuts included in the reference image, namely the reference wheel nut, and determining the current wheel nut at the same position as the reference wheel nut based on the acquired captured image for the accepted reference position. The step of detecting looseness of the current wheel nut by detecting the rotation of the current wheel nut relative to the reference wheel nut included in the reference image; as well as The step of notifying the user that the current wheel nut is loose when it is detected that the current wheel nut is loose.

6. A storage medium, characterized in that, It stores programs for causing the computer to perform the following steps: The display unit shows a pre-capture image that is acquired in preparation for capturing images of the vehicle's wheels at a moment before the user instructs the user to take a picture, a pre-display image that overlaps a frame for specifying the position of a reference position for identifying the position of multiple current wheel nuts set on the wheel, and text indicating the part of the wheel that should be used as the reference position to be placed in the frame. The step of acquiring the captured image based on the received instruction to acquire the captured image; The step of accepting the position within the frame as the reference position in the captured image; The step of storing the previously acquired captured images, i.e., the reference images, and the reference positions of the reference images in association; Based on the reference position of the reference image, the steps are as follows: selecting one of the multiple wheel nuts included in the reference image, namely the reference wheel nut, and determining the current wheel nut at the same position as the reference wheel nut based on the acquired captured image for the accepted reference position. The step of detecting looseness of the current wheel nut by detecting the rotation of the current wheel nut relative to the reference wheel nut included in the reference image; as well as The step of notifying the user that the current wheel nut is loose when it is detected that the current wheel nut is loose.

Images