Car wash system and method for determining whether a car's door mirror is open or closed.

Abstract

To accurately determine the open / closed state of a door mirror. [Solution] The control unit (7) of the car wash device (2) has an image acquisition unit (71) that acquires an image of the automobile, a door mirror detection unit (72) that detects the door mirrors from the image, a first open / close determination unit (73) that determines the open / close state of the door mirrors based on the average intensity distribution in the angular direction in a Fourier transform image obtained by performing a two-dimensional Fourier transform on a first region of the image including the detected door mirrors, and a mechanism control unit (78) that controls the operation of the car wash machine main body (4) based on the determination result.

Description

Technical Field

[0001] The present invention relates to a car washing device and a method for determining the opening and closing of a door mirror of an automobile.

Background Art

[0002] There is a conventionally known device for washing a car that can detect a door mirror of an automobile, which is the vehicle to be washed, from an image captured by a camera attached to the car washing device body.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] When using a car washing device, it is necessary to keep an opening / closing type door mirror in a closed state. However, due to the carelessness of the user, there may be a trouble of damaging the door mirror by washing the car with the opening / closing type door mirror in an open state. In view of this situation, an aspect of the present invention aims to realize a car washing device that can accurately detect the opening and closing state of a door mirror of an automobile, which is the vehicle to be washed.

Means for Solving the Problems

[0005] To solve the above problems, one aspect of the present invention provides a car wash machine comprising: a car wash machine body for washing an automobile; an imaging unit for capturing an image of the automobile including door mirrors; and a control unit, wherein the control unit comprises: an image acquisition unit for acquiring the image; a door mirror detection unit for detecting the door mirrors from the image acquired by the image acquisition unit; a first open / close determination unit for determining the open / closed state of the door mirrors based on the average intensity distribution in the angular direction of a Fourier transform image obtained by performing a two-dimensional Fourier transform on a first region of the image that includes the door mirrors detected by the door mirror detection unit; and a mechanism control unit for controlling the operation of the car wash machine body based on the determination result of the first open / close determination unit.

[0006] To solve the above problems, another aspect of the present invention is a method for determining the opening and closing of a door mirror of an automobile, performed by an information processing device, comprising: an image acquisition step of acquiring an image of the automobile including the door mirror; a door mirror detection step of detecting the door mirror from the acquired image; and an opening and closing determination step of determining the opening and closing state of the door mirror based on the average intensity distribution in the angular direction in a Fourier transform image obtained by performing a two-dimensional Fourier transform on a first region of the image that includes the door mirror detected in the door mirror detection step.

[0007] Each aspect of the present invention may be implemented by a computer, in which case a control program for a car wash device that implements the control device by operating the computer as each part (software element) of the control device, and a computer-readable recording medium on which the program is recorded, also fall within the scope of the present invention. [Effects of the Invention]

[0008] According to one aspect of the present invention, a car wash device can be realized that can accurately detect the open / closed state of the door mirrors of an automobile being washed. [Brief explanation of the drawing]

[0009] [Figure 1] This is a schematic side view of the car wash device and a front view of the car wash machine body, illustrating the car wash device according to Embodiment 1. [Figure 2] This is a schematic diagram showing the main parts of the car wash device according to Embodiment 1 as viewed from above. A parked car X is also shown. [Figure 3] This is a block diagram showing the main components of a car wash device according to Embodiment 1. [Figure 4] This figure illustrates the training data used in machine learning to construct a trained model applied to the door mirror detection unit of the car wash device according to Embodiment 1. It shows an example image of the door mirror in the closed position. [Figure 5] This figure illustrates the training data used in machine learning to construct a trained model applied to the door mirror detection unit of the car wash device according to Embodiment 1. It shows an example image of the door mirror in the open position. [Figure 6] This figure illustrates the output result from the door mirror detection unit to which a learned model has been applied in the car wash device according to Embodiment 1. [Figure 7] This diagram illustrates the operation of the second opening / closing determination unit of the car wash device according to Embodiment 2. It shows an example image of the door mirror in the open state. [Figure 8] This diagram illustrates the operation of the second opening / closing determination unit of the car wash device according to Embodiment 2. It shows an example image of the door mirror in the closed state. [Figure 9] This is a block diagram showing the main components of a car wash device according to Embodiment 2. [Figure 10] This is a schematic diagram showing the main parts of a modified car wash system as viewed from above. A parked car X is also shown. [Modes for carrying out the invention]

[0010] [Embodiment 1] <Overview of the car wash system> The following describes in detail a car wash apparatus according to an embodiment of the present invention. Embodiment 1 describes an exemplary car wash apparatus 2 that performs a car wash process to clean an automobile X.

[0011] Figure 1 is a schematic diagram showing the approximate side view 2S of the car wash machine body 4 and remote panel 6, and the approximate front view 4F of the car wash machine body 4, which are part of the car wash device 2 according to Embodiment 1. Figure 2 is a schematic diagram (schematic plan view) of the main part of the car wash device 2 and a car X parked at the car wash position, which is the stopping position during car washing, as seen from above.

[0012] As shown in Figure 1, the car wash device 2 includes a car wash machine body 4 that washes the vehicle to be washed, automobile X. The car wash device 2 further includes a remote panel 6 that acquires the car wash conditions of automobile X by the car wash machine body 4. In the schematic side view 2S, the outline of the remote panel 6 is shown by a dotted line to indicate that the remote panel 6 is located further back from automobile X in the view of the paper.

[0013] As shown in the general front view 4F, the car wash machine body 4 comprises, for example, two left and right frames 8 and a ceiling portion 10 connecting the upper ends of the two frames 8. The car wash machine body 4 has a structure that allows a vehicle X to pass through a space 4S enclosed by the frames 8 and the ceiling portion 10 along the vehicle X's entry direction DA, as shown in the general side view 2S. In this specification, the entry direction DA is the direction from the front 4A to the rear 4B of the car wash machine body 4.

[0014] As shown in Figure 2, an entry guide G is formed within the car wash area where the car wash machine body 4 is installed. The entry guide G is formed on the ground within the car wash area along the entry direction DA and functions as a reference to indicate where and in what direction a vehicle X to be washed by the car wash machine body 4 should stop. In other words, the entry guide G serves as a guide for at least one of the entry direction DA of the vehicle X to the car wash machine body 4 or the stopping position during washing. The entry guide G includes, for example, a right-side entry guide GR positioned to the right of the ideal stopping position of the vehicle X, and a left-side entry guide GL positioned to the left.

[0015] Furthermore, the car wash device 2 may include a display unit 45 that gives various instructions and / or displays to the driver. The display unit 45 may include, for example, a first display panel 45A installed on the front surface 4A of the car wash machine main body 4 and a second display panel 45B installed behind the car wash machine main body 4 at a position where it can be seen by the driver of the automobile X being washed. Alternatively, the car wash device 2 may be configured to include either the first display panel 45A or the second display panel 45B.

[0016] <car wash machine main body> The car wash machine main body 4 has wheels 12 at each lower part of the frame 8, and by rotating the wheels 12 by a drive unit (not shown), it moves relative to the automobile X in the front-rear direction along the rail R arranged on the ground G. The rail R is formed, for example, along the entry direction DA. Here, while the car wash machine main body 4 is moving relative to the automobile X, it performs cleaning on the automobile X in the space 4S.

[0017] As shown in FIG. 1, of the moving directions of the car wash machine main body 4, the direction toward the front surface 4A side is defined as the forward direction D1, and the direction toward the rear surface 4B side is defined as the backward direction D2. Hereinafter, the movement of the car wash machine main body 4 in the forward direction D1 may be simply referred to as the forward movement of the car wash machine main body 4, and the movement of the car wash machine main body 4 in the backward direction D2 may be simply referred to as the backward movement of the car wash machine main body 4.

[0018] As one of the cleaning parts, the car wash machine main body 4 includes a plurality of brushes that slide on the automobile X and perform brushing. For example, the brushes included in the car wash machine main body 4 include a top brush 14, a side brush 16, and a rocker brush 18 that are each rotated by a rotation motor (not shown). The top brush 14 slides along the upper surface of the automobile X and cleans the upper surface of the automobile X. The left and right side brushes 16 and the rocker brush 18 clean both side surfaces of the automobile X.

[0019] A tank storage section 20 is provided on the side of the car wash machine body 4 to house multiple storage tanks (not shown) containing various liquids, including detergent or wax. Above the tank storage section 20 is a distribution piping section 22 for distributing water, including city water, or liquids from each storage tank. Multiple nozzles, as detailed below, are led out from the distribution piping section 22 via solenoid valves (not shown). These multiple nozzles include a first nozzle that sprays city water or a liquid containing cleaning solution onto the car X to wash the car X, and a second nozzle that sprays a coating agent, including a water-repellent coating agent or wax, onto the car X to form a coating film on the surface of the car X.

[0020] The first nozzle includes a first water purification nozzle 24, a second water purification nozzle 26, a first detergent nozzle 28, and a second detergent nozzle 30. The first water purification nozzle 24 and the second water purification nozzle 26 are positioned on the front 4A and rear 4B sides of each frame 8 of the car wash machine body 4, respectively, and spray water containing tap water onto the car X. The first detergent nozzle 28 and the second detergent nozzle 30 are positioned on the front 4A and rear 4B sides of each frame 8, respectively, and spray a cleaning solution containing shampoo, etc., onto the car X.

[0021] The second nozzle includes a water-repellent coating nozzle 32 and a wax nozzle 34. The water-repellent coating nozzle 32 and the wax nozzle 34 are located on the rear surface 4B of the car wash machine body 4. The water-repellent coating nozzle 32 sprays a water-repellent coating liquid onto the car X. The wax nozzle 34 sprays wax onto the car X.

[0022] Furthermore, the car wash machine body 4 is equipped with a blower 36 that generates airflow to dry the car X. The blower 36 is connected to a top air blower nozzle 38 and side air blower nozzles 40. The top air blower nozzle 38 is located in the upper center of the car wash machine body 4 and blows air toward the ceiling surface of the car X. The side air blower nozzles 40 are located on both sides of the car wash machine body 4 and blow air toward the sides of the car X. The car wash machine body 4 dries the car X after washing by the air blown by the top air blower nozzle 38 and the side air blower nozzles 40.

[0023] The car wash machine body 4 further includes a sensor 52. The sensor 52 is located, for example, on the front side 4A of the car wash machine body 4, closer to the side brushes 16. The sensor 52 is for acquiring information about the external shape of the car X to be washed. More specifically, the sensor 52 can detect the height of the car X as it crosses a predetermined point.

[0024] For example, the sensor 52 may be a multi-axis optical sensor in which a light-emitting part and a light-receiving part are arranged in pairs on the front side and space 4S side of the frame 8 of the car wash machine body 4, with each of the two frames 8 having a light-emitting part. In this multi-axis optical sensor, the individual optical sensors are arranged so that multiple optical axes are aligned in the vertical direction, and each optical axis is on a substantially horizontal plane.

[0025] Therefore, each optical axis sensor in the multi-axis optical axis sensor is configured to detect the presence or absence of automobile X at different heights. In this case, the car wash device 2 can acquire information about the external shape of automobile X by detecting the height at each position across the sensor 52 of automobile X as the car wash machine body 4 moves relative to automobile X from the front to the rear.

[0026] In Figures 1 and 2, for the sake of simplicity, illustrations of the various devices for washing the automobile X provided by the car wash machine body 4 described above may be omitted. Furthermore, the devices provided by the car wash machine body 4 shown in Figure 1 are merely examples, and the car wash machine body 4 may also be equipped with devices for washing the automobile X, including conventionally known configurations, and devices to assist in said washing, on the frame 8 or ceiling 10, in addition to the devices described above.

[0027] An operation panel 42 is located on the front of one frame 8 of the car wash machine body 4. The operation panel 42 is equipped with operation buttons (not shown) for setting car wash conditions. For example, a driver who has gotten out of car X, or another technician, may operate the operation buttons to set car wash conditions, etc.

[0028] <Remote Panel> The remote panel 6 is located, for example, on the front side of the car wash machine body 4 and is positioned roughly in line with the direction of movement of the car wash machine body 4. Furthermore, as shown in Figure 1, the front of the remote panel 6 is positioned to face the side of the car X before it is washed by the car wash machine body 4, in other words, before it enters the interior of the car wash machine body 4.

[0029] As shown in Figure 1, the remote panel 6 comprises a housing 46 and support columns 48 erected on the ground G to support the housing 46. The remote panel 6 may acquire at least a portion of the car washing conditions of the car wash machine body 4 by operation by a driver, such as a touch panel or buttons (not shown), provided on the housing 46.

[0030] <Photography Department> An imaging unit 9 is fixed to the car wash machine body 4. The imaging unit 9 is fixed to the car wash machine body 4 so as to capture images from above the door mirror Xd. The imaging unit 9 may include an imaging unit 9R provided on the upper right side of the car wash machine body 4 and an imaging unit 9L provided on the upper left side. The imaging unit 9, imaging unit 9R, and imaging unit 9L are each examples of imaging units according to this disclosure. The car wash device 2 according to Embodiment 1 includes at least one of the imaging unit 9R and the imaging unit 9L.

[0031] The imaging area of ​​the imaging unit 9R includes the right side of the vehicle X parked at the designated parking position. Therefore, the imaging unit 9R captures an image that includes the right-side door mirror Xd of the vehicle X. The imaging area of ​​the imaging unit 9L includes the left side of the vehicle X parked at the designated parking position. Therefore, the imaging unit 9L captures an image that includes the left-side door mirror Xd of the vehicle X.

[0032] Embodiment 1 shows an example in which the imaging unit 9 is fixed to the car wash machine body 4, but the arrangement of the imaging unit 9 is not limited to this example. The imaging unit 9 may be installed independently of the car wash machine body 4 as long as it is in a position that can capture an image of either of the door mirrors Xd of the automobile X. For example, imaging units 9R and 9L may be attached to support columns installed on both sides of the car wash machine body 4, respectively.

[0033] <Department Head> Furthermore, the car wash system 2 includes a control unit 7 that performs necessary information processing and controls various parts of the car wash system 2. Figure 3 is a functional block diagram showing the main components of the car wash system 2, focusing on the functions of the control unit 7. The control unit 7 may appropriately transmit and receive information between itself and the car wash machine body 4 or the remote panel 6 to control the car wash machine body 4. The control unit 7 may also exchange information with equipment outside the car wash system 2 via the communication unit 60.

[0034] The control unit 7 controls the movement of the car wash machine body 4 along the rail R and the operation of each part of the car wash machine body 4 as described above, thereby performing the washing of the automobile X by the car wash machine body 4. In this case, the control unit 7 may also control the car wash machine body 4 to wash the automobile X by reflecting the car wash conditions acquired by the operation panel 42 and the remote panel 6. Furthermore, the control unit 7 may determine the open / closed state of the automobile X's door mirrors based on the image captured by the imaging unit 9, and control the car wash machine body 4 to wash the automobile X by reflecting the result.

[0035] As shown in Figure 1, the control unit 7 may be physically located on the car wash machine body 4, or it may be located outside the car wash machine body 4. The control unit 7 is composed of a computer having a processor such as a CPU (Central Processing Unit), and each calculation and control is realized by executing the control program stored in the memory unit 76 on the processor.

[0036] Furthermore, the hardware configuration of the control unit 7 is not limited to a CPU; for example, an appropriate information processing device using an ASIC (Application Specific Integrated Circuit) such as a GPU (Graphics Processing Unit), microprocessor, digital signal processor, microcontroller, or TPU (Tensor Processing Unit), or a combination thereof, can be employed. The hardware configuration of the storage unit 76 can include random access memory, flash memory, a hard disk drive, etc.

[0037] As shown in Figure 3, the control unit 7 of the car wash device 2 according to Embodiment 1 includes the following functional blocks: an image acquisition unit 71, a door mirror detection unit 72, a first open / close determination unit 73, a notification unit 77, and a mechanism control unit 78. The control unit 7 also includes the storage unit 76 described above. The image acquisition unit 71 is a functional block that acquires images captured by the imaging unit 9.

[0038] The door mirror detection unit 72 is a functional block that detects the door mirror Xd from the image captured by the imaging unit 9. In Embodiment 1, an example of such a functional block is shown in which a trained model M constructed by machine learning is applied, but the configuration of the door mirror detection unit 72 is not limited to this example. The first open / closed determination unit 73 is a functional block that determines the open / closed state of the door mirror based on the average intensity distribution in the angular direction of the Fourier transform image obtained by performing a two-dimensional Fourier transform on a first region, which is a part of the image that includes the door mirror Xd detected by the door mirror detection unit 72.

[0039] The mechanism control unit 78 is a functional block that controls the above-mentioned parts of the car wash machine body 4 to cause the car wash machine body 4 to wash the automobile X. The notification unit 77 is a functional block that provides notifications via the display unit 45 to drivers or other technicians using the car wash device 2. The notification unit 77 is also a functional block that provides notifications via the communication unit 60 to external devices such as servers.

[0040] <Trained Model M> Next, we will describe in detail the trained model M constructed by machine learning, which is applied to the car wash device 2 of Embodiment 1. The trained model M is an inference model that detects and outputs the door mirror Xd from an image of the automobile X taken by the imaging unit 9 (at least one of imaging unit 9R and imaging unit 9L), including the door mirror Xd. In the following, we will specifically explain using an image of the right side of the automobile X taken by the imaging unit 9R as an example.

[0041] Here, "door mirror" is a class classified by the trained model M, but in Embodiment 1, the trained model M is an inference model to which only "door mirror" is applied as a class. The trained model M also outputs a score representing the likelihood of the inference result to the detected door mirror Xd. In the following, it is assumed that such a score is calculated as a numerical value in the range of 0 to 1, corresponding to the probability that the detected object is indeed of the class "door mirror".

[0042] In this disclosure, machine learning refers to the general process of automatically constructing detection algorithms based on data. In this disclosure, the trained model M can be to which an algorithm for object detection constructed by machine learning can be applied. A convolutional neural network (CNN) is particularly suitable as a machine learning model for this purpose. This is because CNNs are configured to accurately detect target objects regardless of their position in an image or whether the object is horizontally flipped, etc., by applying convolutional layers and pooling layers.

[0043] As a machine learning model to be applied to the trained model M, an inference model may be applied that detects and outputs bounding boxes Bx surrounding objects of a defined class from an input image. A bounding box is a box that encloses the area in which an object is depicted in machine learning tasks such as object detection and image recognition, and in this application, the sides of the bounding box are in the vertical or horizontal direction of the image.

[0044] The following explanation uses the example of applying a pre-trained model M to an inference model that detects and outputs such bounding boxes Bx. However, the machine learning model applied to the pre-trained model M could also be an inference model that extracts only the region of the door mirror Xd from the input image.

[0045] Figures 4 and 5 illustrate the procedure for constructing a trained model M using machine learning. In Figure 4, the label p41 represents an image of the right side of the automobile X captured by the imaging unit 9R when the door mirror Xd is in the closed position. In Figure 5, the label p51 represents an image of the right side of the automobile X captured by the imaging unit 9R when the door mirror Xd is in the open position.

[0046] Furthermore, the symbol p42 in Figure 4 illustrates the situation in which the position of the door mirror Xd in the image represented by symbol p41 is taught by the bounding box Bt surrounding the door mirror Xd. In the figure of symbol p42, the bounding box Bt is shown on the image of symbol p41, indicating that the position of the door mirror Xd is being taught for the image of symbol p41. The same applies to the symbol p52 in Figure 5.

[0047] As shown in Figures 4 and 5, in the example of Embodiment 1, the image of the automobile X captured by the imaging unit 9 is taken from a bird's-eye view of the side of the automobile X, such that the front-to-back direction of the automobile X is the vertical direction of the image. Therefore, in Figure 4, when the door mirror Xd is in the closed position, the bounding box Bt surrounding the door mirror Xd is a vertically elongated rectangle. On the other hand, in Figure 5, when the door mirror Xd is in the closed position, the bounding box Bt surrounding the door mirror Xd is a horizontally elongated rectangle.

[0048] Numerous training data sets are prepared, each consisting of a training image including the car's door mirror captured by the imaging unit 9, and information on the position of the bounding box Bt surrounding the door mirror Xd in the training image. Preferably, the numerous training data sets include both the closed and open states of the car X's door mirror Xd.

[0049] The training images are not limited to images captured by the imaging unit 9; any images similar to those captured by the imaging unit 9 may be used, as long as they are collected separately. A trained model M is constructed by training a machine learning model, such as a convolutional neural network, with a large amount of the above training data using known methods. In the car wash device 2, the constructed trained model M is stored in the memory unit 76 of the control unit 7.

[0050] In Figure 6, the symbol p61 represents the bounding box Bx output by the trained model M as a result of detecting the door mirror Xd when an image of the right side of automobile X captured by the imaging unit 9R was input. As described above, the trained model M outputs the bounding box Bx with a score indicating the likelihood of detecting the door mirror Xd.

[0051] <Characteristic operation of car wash equipment> The characteristic operations performed by the car wash device 2 according to Embodiment 1 are described in detail below. The vehicle to be washed, automobile X, stops at the parking position for car washing. The imaging unit 9 captures an image of automobile X, including the door mirrors Xd. The image acquisition unit 71 acquires the image of automobile X, including the door mirrors Xd, captured by the imaging unit 9.

[0052] It is preferable that the image acquisition unit 71 captures images after the automobile X has stopped at the stopping position. This is because a considerable number of drivers close their side mirrors after stopping the automobile X at the stopping position when using the car wash device 2. Therefore, it is also preferable that the image acquisition unit 71 captures images after the car wash operation has started, for example, after the brushes have started rotating. Alternatively, it is also preferable that the car wash machine body 4 has started moving forward from its standby position.

[0053] Next, the door mirror detection unit 72 executes the trained model M stored in the memory unit 76 to detect the bounding box Bx surrounding the door mirror Xd from the image acquired by the image acquisition unit 71, as described above. Furthermore, the first open / close determination unit 73 uses the bounding box Bx output by the door mirror detection unit 72 to determine the open / closed state of the door mirror Xd.

[0054] Figures 7 and 8 are diagrams illustrating the procedure for determining the open / closed state in the first open / closed determination unit 73. Figure 7 shows an example where the door mirror Xd is in the open state. Figure 8 shows an example where the door mirror Xd is in the closed state. The first open / closed determination unit 73 determines the open / closed state of the door mirror Xd as follows.

[0055] In Figure 7, the symbol p71 indicates a first region, which is a part of the image that includes the area of ​​the door mirror Xd detected by the door mirror detection unit 72. In the example in Figure 7, the first region is assumed to coincide with the area demarcated by the bounding box Bx in the image. However, the first region does not necessarily have to perfectly coincide with the area of ​​the bounding box Bx.

[0056] For example, the first region may be a region that encompasses the region of bounding box Bx and is slightly larger than the region of bounding box Bx. Conversely, the first region may be a region that is encompassed by the region of bounding box Bx and is slightly smaller than the region of bounding box Bx.

[0057] The first opening / closing determination unit 73 acquires the image captured by the imaging unit 9 from the image acquisition unit 71 and cuts out such a first region from the image based on the detection result of the door mirror detection unit 72. Next, the first opening / closing determination unit 73 applies appropriate preprocessing to the image of the first region represented by the symbol p71. The symbol p72 in Figure 7 is the image of the first region after such preprocessing.

[0058] As shown in the image with code p72, such preprocessing is preferably a process that performs so-called edge extraction. It is also preferable that the preprocessing is combined with grayscale conversion and negative / positive inversion as appropriate. The purpose of such preprocessing is to obtain a clear Fourier transform image for detecting the open / closed state of the door mirror Xd in the subsequent processing, and any appropriate processing is acceptable for this purpose.

[0059] Next, the first opening / closing determination unit 73 applies a two-dimensional fast Fourier transform (2DFFT) to the image of code p72 to obtain a Fourier transform image represented by code p73. The two-dimensional fast Fourier transform is an example of a two-dimensional Fourier transform. In the Fourier transform image of code p73, the horizontal axis represents the horizontal frequency kx (wavenumber) of the image, and the vertical axis represents the vertical frequency ky (wavenumber) of the image.

[0060] In the image with code p72, straight lines such as the edge of the door mirror Xd are represented in the Fourier transform as a superposition of waves of various frequency components (wavenumber components) perpendicular to the extension direction of the line. Therefore, in the Fourier transform image with code p73, a strong signal intensity is obtained on a straight line passing through the origin in the extension direction of the line. In the example in Figure 7, this corresponds to the signal on a straight line rotated approximately 20° from the ky axis in the direction of the kx axis.

[0061] Next, the first opening / closing determination unit 73 calculates the average intensity distribution in the angular direction of the Fourier transform image, as shown by the symbol p74 in Figure 7. The angle on the horizontal axis of the graph symbol p74 in Figure 7 is defined as the rotation angle from the positive direction of the kx axis, with the positive direction of the ky axis being 0°. In other words, the angle on the horizontal axis is the angle when rotated clockwise from the positive direction of the ky axis being 0°.

[0062] In Figure 7, the intensity on the vertical axis of the graph labeled p74 represents the average value of the Fourier component intensity on the radial vector for the angle within the range of the graph. Here, the radial vector is a half-line starting from the origin. The label s1 in the graph labeled p74 represents the angular average of the intensity on the vertical axis of the graph over the entire range from 0° to 180°.

[0063] Furthermore, the symbol s2 in the graph with symbol p74 represents the angle average of the intensity on the vertical axis of the graph in the range of angles from 5° to 35°. Here, the angle range from 5° to 35° corresponds to the angle range in which the Fourier component intensity is large when the door mirror Xd is in the open position. In the real space image before the Fourier transform, this angle range corresponds to the angle in which the direction of straight line extension is from 95° to 125°.

[0064] In Figure 8, which shows an example where the door mirror Xd is in the closed state, the symbols p81 to p84 correspond to the images or graphs corresponding to the symbols p71 to p73 in the example in Figure 7, respectively. In the graph of p84 in Figure 8, a large Fourier component intensity is not obtained in the range of angles from 5° to 35°. In this way, the first open / closed determination unit 73 determines whether the door mirror Xd is in the open or closed state based on whether a large Fourier component intensity is obtained in a specific angular range in the average intensity distribution in the angular direction of the Fourier transform image.

[0065] More specifically, by comparing the angular average S1 of the intensity over the entire angular range (0 to 180°) with the angular average S2 of the intensity over the specific angular range, for example, if the magnitude of S2 / S1 is greater than a predetermined value, it can be determined that the door mirror Xd is in the open state, and if it is less than the predetermined value, it can be determined that it is in the closed state. Thus, according to Embodiment 1, it was possible to correctly determine the open or closed state of the door mirror Xd.

[0066] The angle range used for determination here is merely an example and should be set appropriately depending on the placement and orientation of the imaging unit 9 relative to the vehicle X. In particular, the angle ranges for the image taken from the right side of the vehicle X and the image taken from the left side should be reversed horizontally.

[0067] Furthermore, the predetermined value for the determination here should be set appropriately depending on the placement and orientation of the imaging unit 9 relative to the automobile X. Also, if the score output by the door mirror detection unit 72 by the learned model M is lower than the predetermined value, the first open / closed determination unit 73 may determine that it is not possible to determine the open / closed state, as it is unlikely that the door mirror Xd is being detected correctly.

[0068] Furthermore, when the control unit 7 determines the open / closed state of the door mirror Xd based on both the image of the right side of the automobile X captured by the imaging unit 9R and the image of the left side of the automobile X captured by the imaging unit 9L, the final open / closed determination may be made as follows: If the open / closed determinations of both units coincide, that determination result is taken as the final determination. If a determination is not possible in one unit, the determination result of the other unit is adopted. If the determination results of both units are opposite, the final determination is made to be in the open state as a safety measure.

[0069] The control unit 7 performs car washing according to the determined open / closed state of the door mirror Xd, based on the control of each part of the car wash machine body 4 by the mechanism control unit 78. For example, if it is determined that the door mirror Xd is in the open state, the side brush 16 washes the side of the car X, keeping it further away from the side of the car X than when the door mirror Xd is in the closed state.

[0070] Furthermore, if it is determined that the door mirror Xd is in the open position, the notification unit 77 may immediately notify the driver of this fact through the display unit 45. Alternatively, the notification unit 77 may transmit the result of the determination of the open / closed state and an image to the server via the communication unit 60. By accumulating the image and the determination result information on the server, it becomes possible to detect, for example, cases where a misdetermination of the open / closed state of the door mirror Xd occurs in a particular vehicle model.

[0071] Furthermore, in the car wash device 2, if it is reported by operating the remote panel 6 prior to washing that the vehicle to be washed does not have retractable door mirrors, the device may be configured not to determine the open / closed state of the door mirror Xd as described above.

[0072] [Embodiment 2] Other embodiments of the present invention are described below. For the sake of clarity, components having the same function as those described in the above embodiments will be denoted by the same reference numerals, and their descriptions will not be repeated.

[0073] Figure 9 is a functional block diagram showing the main components of the car wash device 2 according to Embodiment 2, focusing on the functions of the control unit 7. Embodiment 2 differs from Embodiment 1 in that the control unit 7 of the car wash device 2 further includes a second opening / closing determination unit 74 and an integrated determination unit 75. The second opening / closing determination unit 74 is a functional block that determines the open / closed state of the door mirror Xd using a different method than the first opening / closing determination unit 73. The integrated determination unit 75 is a functional block that makes a comprehensive judgment based on the opening / closing determination result from the first opening / closing determination unit 73 and the opening / closing determination result from the second opening / closing determination unit 74 to finally determine the open / closed state of the door mirror Xd.

[0074] The second opening / closing determination unit 74 determines the open / closed state of the door mirror Xd as follows. The second opening / closing determination unit 74 calculates the aspect ratio of the bounding box Bx. Here, for example, the aspect ratio is defined as W / H, where H is the vertical length and W is the horizontal length of the bounding box Bx. As described above, when the door mirror Xd is closed, the bounding box Bx is vertically elongated, and the aspect ratio W / H is small. When the door mirror Xd is open, the bounding box Bx is horizontally elongated, and the aspect ratio W / H is large.

[0075] For example, the second opening / closing determination unit 74 determines that the door mirror Xd is in the closed state if the aspect ratio W / H is 1 or less, and determines that the door mirror Xd is in the open state if the aspect ratio W / H is greater than 1. In this way, the second opening / closing determination unit 74 was able to determine the open or closed state of the door mirror Xd.

[0076] Note that the value of 1 in the second open / close determination unit 74 is an example and should be set appropriately depending on the arrangement and orientation of the imaging unit 9 relative to the automobile X. Furthermore, if the score output by the door mirror detection unit 72 by the learned model M is lower than a predetermined value, the second open / close determination unit 74 may determine that it is unlikely that the door mirror Xd is being detected correctly and that it is not possible to determine the open / closed state.

[0077] The integrated determination unit 75 comprehensively judges the results of the opening / closing determination by the first opening / closing determination unit 73 and the results of the opening / closing determination by the second opening / closing determination unit to determine the open / closed state of the door mirror Xd of automobile X. Such comprehensive determination by the integrated determination unit 75 may be performed as appropriate. For example, as a first example, the comprehensive determination may be performed as follows.

[0078] In Embodiment 1, an example was shown in which the opening / closing status is determined based on the relationship between the value S2 / S1 and a predetermined value in the average intensity distribution in the angular direction of the Fourier transform image calculated by the second opening / closing determination unit 74. In this case, for example, if 1 is applied as the predetermined value, if the calculated value S2 / S1 is sufficiently larger than the predetermined value of 1, it can be said with certainty that it is in the open state. Also, if the calculated value S2 / S1 is sufficiently smaller than the predetermined value of 1, it can be said with certainty that it is in the closed state. On the other hand, if the value S2 / S1 is close to the predetermined value of 1, it is considered that there is a high possibility of misdetermining the status.

[0079] Therefore, if the value S2 / S1 calculated by the first opening / closing determination unit 73 is less than or equal to a first threshold (e.g., 0.6) which is less than a predetermined value of 1, and is greater than or equal to a second threshold (e.g., 1.5) which is less than a predetermined value of 1, the overall determination result shall follow the determination result of the first opening / closing determination unit 73. On the other hand, if the value S2 / S1 calculated by the first opening / closing determination unit 73 is greater than the first threshold and less than the second threshold, the overall determination result shall follow the determination result of the second opening / closing determination unit 74.

[0080] Alternatively, as a second example, a comprehensive determination may be made as follows: When the second opening / closing determination unit 74 makes an opening / closing determination with a threshold of 1 for the aspect ratio W / H, if the calculated aspect ratio W / H is sufficiently smaller than the threshold of 1, it can be said with certainty that it is in the open state. Also, if the aspect ratio H / W calculated by the second opening / closing determination unit 74 is sufficiently larger than the threshold of 1, it can be said with certainty that it is in the closed state.

[0081] Therefore, normally, the overall determination result will follow the determination result of the second opening / closing determination unit 74, provided that the aspect ratio H / W calculated by the second opening / closing determination unit 74 is less than or equal to a first threshold (e.g., 0.6) which is sufficiently smaller than threshold 1, and greater than or equal to a second threshold (e.g., 1.5) which is sufficiently larger than threshold 1.

[0082] [Variation 1] The car wash device 2 according to Modification 1 is a modification of the car wash device 2 according to Embodiment 1 or the car wash device 2 according to Embodiment 2. Modification 1 differs from the above embodiments in that the door mirror detection unit 72 is configured to detect the door mirror Xd from the image captured by the imaging unit 9 using a known pattern matching technique.

[0083] In the modified example 1, the door mirror detection unit 72 detects the door mirror Xd by pattern matching using a pre-registered pattern. The pattern registered to detect the door mirror Xd is an image of the door mirror Xd, or an image from which the features of the door mirror Xd have been extracted, and may also be called a template image in pattern matching technology. To accommodate various door mirrors Xd, the door mirror detection unit 72 is equipped with a variety of registered patterns.

[0084] The operation of each functional block of the control unit 7 of the car wash device 2 according to Modification 1, other than the door mirror detection unit 72, is the same as in the above embodiments. In this way, the car wash device 2 according to Modification 1 can detect the open / closed state of the door mirror Xd in the same manner as in the above embodiments.

[0085] [Variation 2] The car wash device 2 according to Modification 2 is a modified version of the car wash device 2 according to Embodiment 1 or the car wash device 2 according to Embodiment 2. In Modification 2, the mounting position and shooting direction of the imaging unit 9 on the car wash machine body 4 are different from those shown in Figure 2.

[0086] Figure 10 is a schematic diagram showing the main parts of the car wash device 2 according to Modification 2 when viewed from above, and shows the mounting position and shooting direction of the imaging unit 9 in Modification 2. In Figure 2, the imaging unit 9 captured images at an angle that captured the sides of the automobile X. However, in Modification 2, in the horizontal direction, the imaging unit 9 captures images of the automobile X outwards to the left and right of the automobile at an angle.

[0087] Therefore, in Modification 2, the imaging unit 9 captures an image of the car X, including the door mirror Xd, from above at an angle that makes it difficult for the side of the car X, which is parked at the parking position, to be captured. According to Modification 2, since the side of the car X is less likely to be captured in the image of the car X captured by the imaging unit 9, the detection accuracy of the door mirror Xd by the door mirror detection unit 72 (trained model M) may be further improved.

[0088] 〔summary〕 Aspect 1 of the present disclosure is a car wash machine that washes an automobile, an imaging unit that captures an image of the automobile including a door mirror, and a control unit, wherein the control unit includes an image acquisition unit that acquires the image, a door mirror detection unit that detects the door mirror from the image acquired by the image acquisition unit, a first open / close determination unit that determines the open / closed state of the door mirror based on the average intensity distribution in the angular direction of a Fourier transform image obtained by performing a two-dimensional Fourier transform on a first region of the image that includes the door mirror detected by the door mirror detection unit, and a mechanism control unit that controls the operation of the car wash machine based on the determination result of the first open / close determination unit.

[0089] Aspect 2 of the present disclosure is a car wash apparatus according to aspect 1, wherein the first area is an area demarcated by a bounding box surrounding the door mirror, the control unit further includes a second opening / closing determination unit that determines the opening / closing state of the door mirror based on the aspect ratio of the first area, and the mechanism control unit may be configured to control the operation of the car wash machine body based on the determination result of the first opening / closing determination unit and the determination result of the second opening / closing determination unit.

[0090] Aspect 3 of this disclosure is a car wash apparatus according to aspect 1 or 2, wherein the door mirror detection unit detects the door mirror using a trained model constructed by machine learning, and the trained model may be constructed by machine learning using training data consisting of a set of training images which are images of an automobile that include the door mirror and information on the position of the door mirror in the training images.

[0091] Aspect 4 of the present disclosure is a car wash apparatus according to aspect 3, wherein the position information of the door mirror in the learning image is represented as information of a bounding box surrounding the door mirror, and the first region in the image is configured to be a region partitioned by a bounding box surrounding the door mirror, as detected by the door mirror detection unit.

[0092] Aspect 5 of the present disclosure is a car wash device according to aspect 1 or 2, wherein the door mirror detection unit is configured to detect the bounding box by detecting the door mirror through pattern matching using a pre-registered pattern indicating the door mirror.

[0093] Aspect 6 of the present disclosure is a control program for causing a computer to function as a control unit for a car wash apparatus according to any of aspects 1 to 5, wherein the computer functions as the image acquisition unit, the door mirror detection unit, the first opening / closing determination unit, and the mechanism control unit.

[0094] Aspect 7 of the present disclosure is a method for determining the opening and closing of a door mirror of an automobile, performed by an information processing device, comprising: an image acquisition step of acquiring an image of the automobile including the door mirror; a door mirror detection step of detecting the door mirror from the acquired image; and an opening and closing determination step of determining the opening and closing state of the door mirror based on the average intensity distribution in the angular direction in a Fourier transform image obtained by performing a two-dimensional Fourier transform on a first region of the image that includes the door mirror detected in the door mirror detection step.

[0095] Aspect 8 of the present disclosure is a car wash device according to any of aspects 1 to 5, wherein the imaging unit may be fixed to the car wash machine body so as to capture the image from above the door mirror.

[0096] The present invention is not limited to the embodiments described above, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of the present invention. [Explanation of symbols]

[0097] 2 Car wash equipment 4 Car wash machine body 6. Remote Panel 7 Control Unit 71 Image acquisition unit 72 Door mirror detection unit 73 First Opening / Closing Determination Unit 74 Second Opening / Closing Determination Unit 75 Integrated Determination Unit 76 Memory section 77 Hochi Department 78 Mechanism Control Unit 9, 9R, 9L Imaging Unit 45 Display section 60 Communications Department X car Xd Door Mirrors Bt bounding box (training data) Bx Bounding Box

Claims

1. The car wash machine itself, An imaging unit that captures an image of the automobile, including the door mirrors, A car wash device comprising a control unit, The control unit, An image acquisition unit that acquires the aforementioned image, From the image acquired by the image acquisition unit, a door mirror detection unit detects the door mirror, A first open / close determination unit determines the open / closed state of the door mirror based on the average intensity distribution in the angular direction of the Fourier transform image obtained by performing a two-dimensional Fourier transform on a first region, which is a part of the aforementioned image and includes the door mirror detected by the door mirror detection unit. A car wash device having a mechanism control unit that controls the operation of the car wash machine body based on the determination result of the first opening / closing determination unit.

2. A method for determining the opening and closing of an automobile's side mirror, which is performed by an information processing device, Image acquisition step: acquires an image of the automobile including the door mirrors, A door mirror detection step is performed to detect the door mirror from the acquired image, A method for determining the open / closed state of a door mirror of an automobile, comprising: an open / closed determination step, which determines the open / closed state of the door mirror based on the average intensity distribution in the angular direction in a Fourier transform image obtained by performing a two-dimensional Fourier transform on a first region that includes the door mirror detected in the door mirror detection step, which is a part of the aforementioned image.

Images