Information processing device, determination program, determination method, and car wash device

Abstract

Improve the accuracy of equipment detection. [Solution] An information processing device (70) for determining whether or not a second piece of equipment (FP) is installed in an automobile includes: an image acquisition unit (71) for acquiring a primary image; a detection unit (72) for detecting the position of the first piece of equipment by inputting the primary image into a first trained model; a secondary image generation unit (73) for generating a secondary image based on position information indicating the position of the first piece of equipment; and a determination unit (74) for determining whether or not the second piece of equipment is included in the secondary image by inputting the secondary image into a second trained model.

Description

Technical Field

[0001] The present disclosure relates to an information processing device, a determination program, a determination method, and a car wash device for determining whether a second equipment is installed in a vehicle.

Background Art

[0002] As in the car wash machine described in Patent Document 1, a car wash machine equipped with equipment position detection means for extracting equipment of a vehicle from an image captured by a camera attached to the car wash machine body and calculating the position of the equipment is known as the prior art.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, among the equipment, there is equipment with low detection accuracy by existing detection means.

[0005] One aspect of the present disclosure aims to accurately detect equipment.

Means for Solving the Problems

[0006] An information processing device according to one aspect of the present disclosure is an information processing device for determining whether or not a second piece of equipment is installed in an automobile, and includes: an image acquisition unit that acquires an image including the automobile as a primary image; a detection unit that detects the position of a first piece of equipment by inputting the primary image to a first trained model that has been trained to detect a first piece of equipment which is essential equipment in the automobile, using the image including the automobile as input information; a secondary image generation unit that generates a secondary image from the primary image, based on position information indicating the position of the first piece of equipment detected by the detection unit, including a region in which the second piece of equipment is located if it is assumed that the automobile is equipped with the second piece of equipment; and a determination unit that determines whether or not the second piece of equipment is included in the secondary image by inputting the secondary image to a second trained model that has been trained to detect a second piece of equipment provided in the automobile, using the image including the automobile as input information.

[0007] A determination program according to one aspect of this disclosure is a determination program for causing a computer to function as the above-mentioned information processing device, and is a determination program for causing the computer to function as the image acquisition unit, the detection unit, the secondary image generation unit, and the determination unit.

[0008] A determination method according to one aspect of the present disclosure includes: an image acquisition step of acquiring an image including an automobile as a primary image; a detection step of detecting the position of a first piece of equipment by inputting the primary image to a first trained model that has been trained to detect a first piece of equipment provided by the automobile, using the image including the automobile as input information; a secondary image generation step of generating a secondary image from the primary image, based on position information indicating the position of the first piece of equipment detected in the detection step, which includes a region where a second piece of equipment would be located if the automobile were equipped with a second piece of equipment; and a determination step of determining whether or not the second piece of equipment is included in the secondary image by inputting the secondary image to a second trained model that has been trained to detect a second piece of equipment provided by the automobile, using the image including the automobile as input information.

[0009] A car wash system according to one aspect of the present disclosure comprises a car wash machine body for washing an automobile, an imaging unit for capturing an image including the automobile, an information processing device according to any one of claims 1 to 6, and a control unit for controlling the operation of the car wash machine body based on the determination result of the determination unit.

[0010] Each aspect of the information processing device described herein may be implemented by a computer. In this case, the program for the information processing device that enables the computer to implement the information processing device by operating the computer as each part (software element) of the information processing device, and the computer-readable recording medium on which the program is recorded, also fall within the scope of this disclosure. [Effects of the Invention]

[0011] According to one aspect of this disclosure, equipment can be detected with high accuracy. [Brief explanation of the drawing]

[0012] [Figure 1] A functional block diagram showing the main components of a car wash system according to Embodiment 1 of this disclosure. [Figure 2] This is a schematic diagram showing the general side and general front views of the car wash device according to Embodiment 1. [Figure 3] This is a flowchart showing the processing flow executed by the information processing unit according to Embodiment 1. [Figure 4] This shows an image captured by the imaging device according to Embodiment 1. [Figure 5] This figure shows the bounding box output by the first model according to Embodiment 1 as a result of detecting the headlights. [Figure 6] This shows an image generated by the secondary image generation unit according to Embodiment 1. [Figure 7] This figure shows the bounding box output by the second model according to Embodiment 1 as a result of detecting the fender pole. [Figure 8]It is a flowchart showing another example of the processing flow executed by the information processing unit according to Embodiment 1.

Mode for Carrying Out the Invention

[0013] In the following Embodiment 1, an example in which the information processing apparatus according to the present disclosure is applied to a car wash system will be described.

[0014] 〔Embodiment 1〕 Hereinafter, the car wash system 100 according to Embodiment 1 will be described.

[0015] FIG. 1 is a functional block diagram showing the main configuration of the car wash system 100. As shown in FIG. 1, the car wash system 100 includes a car wash apparatus 2, an imaging device 9, and a learning device 80. The car wash system 100 is a system that controls the car wash machine body so as to execute a car wash process for washing the automobile X. The car wash apparatus 2 includes a control unit 7 that controls each part of the car wash machine body 4 and an information processing unit 70 corresponding to the information processing apparatus according to the present disclosure.

[0016] In the car wash system 100, the car wash apparatus 2, the learning device 80, and the imaging device 9 can be communicably connected to each other. For example, the car wash apparatus 2 and each device can be connected via a network. In this case, the network may be a wired LAN (Local Area Network), a wireless LAN, the Internet, a public switched telephone network, a mobile data communication network, or a combination thereof.

[0017] In Embodiment 1, as an example, an example in which the information processing unit 70 determines the presence or absence of fender poles will be described. Also, in Embodiment 1, an example in which a headlight is used as an essential equipment item in an automobile for determining the presence or absence of fender poles will be described. That is, in Embodiment 1, the fender pole corresponds to the second equipment item according to the present disclosure, and the headlight corresponds to the first equipment item according to the present disclosure.

[0018] (Functional Configuration of Learning Device) As shown in FIG. 1, the learning device 80 includes a control unit 81 and a storage unit 82. The control unit 81 includes a first learning unit 81A and a second learning unit 81B.

[0019] The first learning unit 81A is a first model M1 that takes an image including an automobile as an input, and generates the first model M1 used for detecting a headlight by machine learning. Details of the first model M1 will be described later. When the first learning unit 81A receives an image from the information processing unit 70, the first learning unit 81A inputs the image into the first model M1, and transmits the information output from the first model M1 to the information processing unit 70. The first model M1 is an example of the first learned model according to the present disclosure. In the present disclosure, the "image including an automobile" may be an image including an automobile as a subject. Further, the "image including an automobile" may be an image including at least a part of an automobile.

[0020] The second learning unit 81B is a second model M2 that takes an image including an automobile as an input, and generates the second model M2 used for detecting a fender pole by machine learning. Details of the second model M2 will be described later. When the second learning unit 81B receives an image from the information processing unit 70, the second learning unit 81B inputs the image into the second model M2, and transmits the information output from the second model M2 to the information processing unit 70. The second model M2 is an example of the second learned model according to the present disclosure.

[0021] The storage unit 82 stores various data referred to by the control unit 81. The storage unit 82 also stores the first model M1 and the second model M2 generated by the first learning unit 81A and the second learning unit 81B.

[0022] The learning device 80 may be a server arranged on a network, and may be connected to a plurality of car wash devices and imaging devices.

[0023] (Generation of the first model M1) Here, the generation of the first model M1 will be described. The first learning unit 81A of the learning device 80 acquires multiple images to be used as training data. The acquired images include automobiles, and also include headlights.

[0024] For example, the training data may consist of a set of training images, including the headlights of a car, captured by the imaging device 9, and an image to which a bounding box surrounding the headlights and a label indicating "headlights" have been added. In other words, the training data may include information indicating the type of the first equipment, in addition to information indicating the portion of the bounding box corresponding to the first equipment. The information indicating the portion of the bounding box may be, for example, the coordinates of the bounding box in the image. Note that the training images are not limited to images captured by the imaging device 9; images similar to those captured by the imaging device 9 may be collected separately. For example, the training images may be images of cars that are not being washed, captured by an imaging device located in a different place from the car wash device 2.

[0025] A first model M1 is constructed by training a machine learning model, such as a convolutional neural network, with a large amount of the above-mentioned training data using known methods. The constructed first model M1 can be stored in the memory unit 82 of the learning device 80.

[0026] (Generation of the second model M2) The second learning unit 81B of the learning device 80 acquires multiple images to be used as training data. The acquired images include automobiles, and also include fender poles.

[0027] For example, the training data may consist of a set of training images, including a car fender pole captured by the imaging device 9, and an image to which a bounding box surrounding the fender pole and a label indicating that it is a "fender pole" have been added. In other words, the training data may include information indicating the type of the second piece of equipment, in addition to information indicating the bounding box portion corresponding to the second piece of equipment. Similar to the first model M1, the training images for the second model M2 are not limited to images captured by the imaging device 9; images similar to those captured by the imaging device 9 may be used, as well as images collected separately.

[0028] A second model M2 is constructed by training a machine learning model, such as a convolutional neural network, with a large amount of the above-mentioned training data using known methods. The constructed second model M2 can be stored in the memory unit 82 of the learning device 80.

[0029] <Imaging device> In this embodiment, the imaging device 9 captures an image including the automobile X that is to be washed. More specifically, it captures an image including a first piece of equipment that is essential equipment on automobile X (e.g., headlights) and a second piece of equipment that the presence or absence of on automobile X is to be determined (e.g., fender poles).

[0030] In this disclosure, the first equipment is an essential piece of equipment for the vehicle, and the second equipment may be an optional piece of equipment. If the second equipment is an optional piece of equipment, the vehicle X may not be equipped with the second equipment. In that case, the image captured by the imaging device 9 should include the area where the fender pole is located, assuming that the fender pole is equipped.

[0031] In Embodiment 1, the imaging device 9 is fixed to the upper front of the car wash machine body 4, as shown in the schematic front view 4F of Figure 2. The imaging device 9 may include an imaging device 9R provided on the upper right side of the car wash machine body 4 and an imaging device 9L provided on the upper left side. Imaging devices 9R and 9L are examples of imaging devices according to the present disclosure. The car wash machine 2 according to Embodiment 1 includes at least one of the imaging device 9R and the imaging device 9L.

[0032] The imaging area of ​​imaging device 9R includes the right side of vehicle X parked at the parking position. Therefore, imaging device 9R captures an image that includes the right headlight HL of vehicle X and the fender pole FP (or the position where the fender pole FP should be located). The imaging area of ​​imaging device 9L includes the left side of vehicle X parked at the parking position. Therefore, imaging device 9L captures an image that includes the left headlight HL of vehicle X and the fender pole FP (or the position where the fender pole FP should be located).

[0033] Embodiment 1 shows an example in which the imaging device 9 is fixed to the car wash machine body 4, but the arrangement of the imaging device 9 is not limited to this example. The imaging device 9 may be installed independently of the car wash machine body 4 as long as it is in a position to capture images of either the headlight HL and fender pole FP of the automobile X. For example, imaging devices 9R and 9L may be attached to support columns installed on both sides of the car wash machine body 4, respectively. Alternatively, it may be attached to a vehicle height gate 49 that is supported by a support column 49A erected on the side of the access road and installed in front of the remote panel 6. The schematic side view 2S of Figure 2 shows an example in which the imaging device 9G is attached to the vehicle height gate 49.

[0034] Furthermore, the imaging device 9 may be installed in an appropriate position to capture images including the first and second equipment, depending on the types of the first and second equipment. If the first and second equipment are located at the rear of the vehicle, the imaging device 9 may be installed in a position to capture images from the rear of the vehicle. For example, if the car wash machine body 4 is installed indoors, the imaging device 9 may be installed on a beam or the like of a building surrounding the car wash machine body 4 in order to capture images from the rear of the vehicle.

[0035] <Overview of the car wash system> Figure 2 is a schematic diagram showing the approximate side view 2S of the car wash machine body 4 and the remote panel 6, and the approximate front view 4F of the car wash machine body 4, which are included in the car wash device 2 according to Embodiment 1.

[0036] As shown in Figures 1 and 2, the car wash device 2 includes a car wash machine body 4 that washes the vehicle to be washed, which is vehicle X. The car wash device 2 further includes a remote panel 6 and a vehicle height gate 49 that acquire the car wash conditions of vehicle X by the car wash machine body 4. In the schematic side view 2S, the outline of the remote panel 6 is shown by a dotted line to indicate that the remote panel 6 is located further back from vehicle X in the plane of the paper.

[0037] As shown in the general front view 4F, the car wash machine body 4 comprises, for example, two left and right frames 8 and a ceiling portion 10 connecting the upper ends of the two frames 8. The car wash machine body 4 has a structure that allows a vehicle X to pass through a space 4S enclosed by the frames 8 and the ceiling portion 10 along the vehicle X's entry direction DA, as shown in the general side view 2S. In this specification, the entry direction DA is the direction from the front 4A to the rear 4B of the car wash machine body 4.

[0038] <Car wash machine body> The car wash machine body 4 comprises a drive unit 4X and a cleaning unit 4Y. The drive unit 4X comprises wheels 12 provided at the bottom of each frame 8 and a drive device (not shown) that rotates the wheels 12. As the drive device rotates the wheels 12, the car wash machine body 4 moves relative to the automobile X in the front-rear direction along rails R arranged on the ground G. The rails R are formed, for example, along the entry direction DA. While moving relative to the automobile X, the car wash machine body 4 performs washing (cleaning) of the automobile X in the space 4S.

[0039] As shown in Figure 2, the direction of movement of the car wash machine body 4 toward the front 4A is defined as the forward direction D1, and the direction toward the rear 4B is defined as the reverse direction D2. Hereafter, when the car wash machine body 4 moves in the forward direction D1, it may simply be referred to as the car wash machine body 4 moving forward, and when the car wash machine body 4 moves in the reverse direction D2, it may simply be referred to as the car wash machine body 4 moving backward.

[0040] The car wash machine body 4 is equipped with multiple brushes that slide and brush over the car X as one of the cleaning units 4Y. For example, the brushes provided by the car wash machine body 4 include a top brush 14, side brushes 16, and rocker brushes 18, each rotated by a rotary motor (not shown). The top brush 14 slides along the top surface of the car X and cleans the top surface of the car X. The left and right side brushes 16 and rocker brushes 18 clean both sides of the car X.

[0041] A tank storage section 20 is provided on the side of the car wash machine body 4 to house multiple storage tanks (not shown) containing various liquids, including detergent or wax. Above the tank storage section 20 is a distribution piping section 22 for distributing water, including city water, or liquids from each storage tank. Multiple nozzles included in the cleaning section 4Y are led out from the distribution piping section 22 via solenoid valves (not shown). These multiple nozzles include a first nozzle that sprays a liquid containing city water or cleaning solution onto the car X to wash the car X, and a second nozzle that sprays a coating agent containing a water-repellent coating agent or wax onto the car X to form a coating film on the surface of the car X.

[0042] The first nozzle includes a first water purification nozzle 24, a second water purification nozzle 26, a first detergent nozzle 28, and a second detergent nozzle 30. The first water purification nozzle 24 and the second water purification nozzle 26 are positioned on the front 4A and rear 4B sides of each frame 8 of the car wash machine body 4, respectively, and spray water containing tap water onto the car X. The first detergent nozzle 28 and the second detergent nozzle 30 are positioned on the front 4A and rear 4B sides of each frame 8, respectively, and spray a cleaning solution containing shampoo, etc., onto the car X.

[0043] The second nozzle includes a water-repellent coating nozzle 32 and a wax nozzle 34. The water-repellent coating nozzle 32 and the wax nozzle 34 are located on the rear surface 4B of the car wash machine body 4. The water-repellent coating nozzle 32 sprays a water-repellent coating liquid onto the car X. The wax nozzle 34 sprays wax onto the car X.

[0044] Furthermore, the car wash machine body 4 is equipped with a blower 36, which is included in the cleaning unit 4Y and generates airflow to dry the automobile X. The blower 36 is connected to a top air blower nozzle 38 and side air blower nozzles 40. The top air blower nozzle 38 is located at the top center of the car wash machine body 4 and blows air toward the ceiling surface of the automobile X. The side air blower nozzles 40 are located on both sides of the car wash machine body 4 and blow air toward the sides of the automobile X. The car wash machine body 4 dries the automobile X after washing by the air blown by the top air blower nozzle 38 and the side air blower nozzles 40.

[0045] The car wash machine body 4 further includes a sensor 52. The sensor 52 is located, for example, on the front side 4A of the car wash machine body 4, closer to the side brushes 16. The sensor 52 is for acquiring information about the external shape of the car X to be washed. More specifically, the sensor 52 can detect the height of the car X as it crosses a predetermined point.

[0046] For example, the sensor 52 may be a multi-axis optical sensor in which a light-emitting part and a light-receiving part are arranged in pairs on the front side and space 4S side of the frame 8 of the car wash machine body 4, with each of the two frames 8 having a light-emitting part. In this multi-axis optical sensor, the individual optical sensors are arranged so that multiple optical axes are aligned in the vertical direction, and each optical axis is on a substantially horizontal plane.

[0047] Therefore, each optical axis sensor in the multi-axis optical axis sensor is configured to detect the presence or absence of automobile X at different heights. In this case, the car wash device 2 can acquire information about the external shape of automobile X by detecting the height at each position across the sensor 52 of automobile X as the car wash machine body 4 moves relative to automobile X from the front to the rear.

[0048] In Figure 2, for the sake of simplicity, the illustration of the various devices for washing the automobile X provided by the car wash machine body 4 described above may be omitted. Furthermore, the devices provided by the car wash machine body 4 shown in Figure 2 are merely examples, and the car wash machine body 4 may also be equipped with devices for washing the automobile X, including conventionally known configurations, and devices to assist in said washing, on the frame 8 or ceiling 10, in addition to the devices described above.

[0049] An operation panel 42 is located on the front of one frame 8 of the car wash machine body 4. The operation panel 42 is equipped with operation buttons (not shown) for setting car wash conditions. For example, a driver who has gotten out of car X, or another technician, may operate the operation buttons to set car wash conditions, etc.

[0050] The remote panel 6 is located, for example, on the front side of the car wash machine body 4 and is positioned roughly in line with the direction of movement of the car wash machine body 4. Furthermore, as shown in Figure 2, the front of the remote panel 6 is positioned to face the side of the car X before it is washed by the car wash machine body 4, in other words, before it enters the interior of the car wash machine body 4.

[0051] As shown in Figure 2, the remote panel 6 comprises a housing 46 and support columns 48 erected on the ground G to support the housing 46. The remote panel 6 may acquire at least a portion of the car washing conditions of the car wash machine body 4 by operation by a driver, such as a touch panel or buttons (not shown), provided on the housing 46.

[0052] As shown in Figure 2, the vehicle height gate 49 is supported by a support column 49A erected on the side of the access road and is installed in front of the remote panel 6. The vehicle height gate 49 determines the maximum vehicle height of the vehicle X and prevents vehicles X that exceed the maximum vehicle height from entering.

[0053] <Department Head> The car wash system 2 includes a control unit 7 that controls various parts of the car wash system 2. The control unit 7 may appropriately transmit and receive information between itself and the car wash machine body 4 or the remote panel 6, and control the car wash machine body 4. The control unit 7 may also exchange information with equipment outside the car wash system 2 via the communication unit 60.

[0054] The control unit 7 includes a mechanism control unit 7X. The mechanism control unit 7X is a functional block that controls the above-mentioned parts of the car wash machine body 4 to cause the car wash machine body 4 to wash the automobile X. Specifically, the control unit 7 controls the movement of the car wash machine body 4 along the rail R by controlling the drive unit 4X, and washes the automobile X by controlling the cleaning unit 4Y. In this case, the control unit 7 may also control the car wash machine body 4 to wash the automobile X by reflecting the car wash conditions acquired by the operation panel 42 and the remote panel 6. Furthermore, the control unit 7 may also control the car wash machine body 4 to wash the automobile X by reflecting the results determined by the information processing unit 70, which will be described later.

[0055] As shown in Figure 2, the control unit 7 may be physically located on the car wash machine body 4, or it may be located outside the car wash machine body 4. The control unit 7 is composed of a computer having a processor such as a CPU (Central Processing Unit), and each calculation and control is realized by executing a control program stored in a memory unit (not shown) on the processor.

[0056] Furthermore, the hardware configuration of the control unit 7 is not limited to a CPU; for example, an appropriate information processing device using an ASIC (Application Specific Integrated Circuit) such as a GPU (Graphics Processing Unit), microprocessor, digital signal processor, microcontroller, or TPU (Tensor Processing Unit), or a combination thereof, can be employed. The hardware configuration of the storage unit can include random access memory, flash memory, a hard disk drive, etc.

[0057] <Information Processing Department> Furthermore, the car wash device 2 includes an information processing unit 70 that performs the necessary information processing. As shown in Figure 1, the information processing unit 70 includes an image acquisition unit 71, a detection unit 72, a secondary image generation unit 73, a determination unit 74, and a identification unit 75.

[0058] The image acquisition unit 71 is a functional block that acquires an image including the automobile X captured by the imaging device 9 as a primary image.

[0059] The detection unit 72 is a functional block that detects the position of the first equipment in a primary image by inputting the primary image into a first trained model that has been trained to detect the first equipment, which is an essential piece of equipment on the automobile, using an image including the automobile as input information.

[0060] The secondary image generation unit 73 is a functional block that generates a secondary image from the primary image based on position information indicating the position of the first equipment detected by the detection unit 72, including the region where the second equipment would be located if the automobile X were equipped with the second equipment.

[0061] The determination unit 74 is a functional block that determines whether or not a second piece of equipment is included in a secondary image by inputting the secondary image into a second trained model that has been trained to detect a second piece of equipment provided by the automobile, using an image including the automobile as input information.

[0062] The specific unit 75 is a functional block that identifies the position of the second piece of equipment by inputting the secondary image into the second trained model.

[0063] The information processing unit 70 may be located in the car wash machine body 4, or it may be located outside the car wash machine body 4. The information processing unit 70 is composed of a computer having a processor such as a CPU (Central Processing Unit), and various calculations and controls are realized by executing an information processing program stored in a memory unit (not shown) on the processor.

[0064] Furthermore, the hardware configuration of the information processing unit 70 is not limited to a CPU; for example, an appropriate information processing device using an ASIC (Application Specific Integrated Circuit) such as a GPU (Graphics Processing Unit), microprocessor, digital signal processor, microcontroller, or TPU (Tensor Processing Unit), or a combination thereof, can be employed. The hardware configuration of the storage unit can include random access memory, flash memory, a hard disk drive, etc.

[0065] [Example of processing in the first case] In the following, an exemplary process of the information processing unit 70 in this embodiment will be described with reference to Figures 3 to 7. Figure 3 is an exemplary flowchart showing the flow of processing performed by the information processing unit 70.

[0066] In step S1, the image acquisition unit 71 acquires the image P1 captured by the imaging device 9 as the primary image. In other words, step S1 is an example of an image acquisition process in which an image including an automobile is acquired as the primary image. Figure 4 shows the image P1 captured by the imaging device 9L. Image P1 is an image of the left side of the automobile X captured by the imaging device 9L, and is an example of a primary image according to this disclosure.

[0067] In step S2, the detection unit 72 detects the position of the headlight HL in the image P1 acquired by the image acquisition unit 71. In other words, step S2 is an example of a detection process in which the position of a first piece of equipment is detected by inputting a primary image into a first trained model that has been trained to detect a first piece of equipment provided by an automobile, using an image including the automobile as input information.

[0068] In step S2, the detection unit 72 transmits image P1 to the learning device 80. When the first learning unit 81A of the learning device 80 receives image P1 from the detection unit 72, it inputs image P1 to the first model M1 and transmits the information output from the first model M1 to the information processing unit 70.

[0069] Here, we will describe in detail the first model M1, which is built using machine learning and applied to the car wash system 100 of Embodiment 1. The first model M1 is an inference model that detects and outputs a bounding box surrounding the headlights from an input image. In this process, the image input to the first model M1 is an image of the automobile X that includes the headlights, captured by the imaging device 9 (at least one of imaging device 9R and imaging device 9L).

[0070] Here, "headlight" is a class classified by the first model M1. In Embodiment 1, the first model M1 may be an inference model to which only "headlight" is applied as a class. A bounding box is a box that encloses the area in which an object is depicted in machine learning tasks such as object detection and image recognition, using a rectangle or square.

[0071] Furthermore, the first model M1 may output a bounding box surrounding the headlight with a score representing the likelihood of the inference result. Such a score may be a numerical value in the range of 0 to 1, corresponding to the probability that the object indicated by the bounding box is indeed a headlight.

[0072] In this disclosure, machine learning refers to the general process of automatically constructing detection algorithms based on data. In this disclosure, algorithms for object detection constructed by machine learning can be applied to the first model M1 and the second model M2. A convolutional neural network (CNN) is particularly suitable as a machine learning model for this purpose. This is because CNNs are configured to accurately detect target objects regardless of their position in an image or whether the object is horizontally flipped, by applying convolutional layers and pooling layers.

[0073] Figure 5 shows the bounding box Bx output by the first model M1 as a result of detecting a headlight HL when image P1 is input, displayed on the image. The information output from the first model M1 may include a label indicating that the detected object is a headlight and coordinate information of the bounding box Bx. The detection unit 72 receives the information output from the first model M1.

[0074] In step S2, the detection unit 72 generates a bounding box surrounding the first equipment in the primary image through the above-described process, and detects the position of the first equipment in the primary image based on the coordinates of the bounding box. Specifically, the detection unit 72 detects the position of the headlight HL from the image P1, which is determined by the coordinates of the bounding box Bx surrounding the headlight HL.

[0075] In this example, the detection unit 72 is configured with a first model M1 constructed using machine learning; however, the configuration of the detection unit 72 is not limited to this example.

[0076] In step S3, the secondary image generation unit 73 generates a secondary image (image P2) from image P1, including the region where the fender pole is located, based on the coordinate information of the bounding box Bx detected by the detection unit 72. In other words, step S3 is an example of a secondary image generation process in which, based on position information indicating the position of the first equipment detected in the detection step (S2), a secondary image is generated from the primary image, including the region where the second equipment is located, assuming that the automobile is equipped with the second equipment. Figure 6 shows image P2 generated by the secondary image generation unit 73. Image P2 is an example of a secondary image according to this disclosure.

[0077] A secondary image is an image obtained by cutting out (extracting) a portion of a primary image. More specifically, a secondary image may be an image obtained by trimming a primary image. The secondary image generation unit 73 can determine the region to be cut out as a secondary image from the primary image based on the coordinate information of the bounding box Bx. In other words, the secondary image generation unit 73 can determine the coordinates of the region to be cut out as a secondary image based on the coordinate information of the bounding box Bx.

[0078] For example, since the headlight HL and the fender pole FP are located in close proximity, the secondary image generation unit 73 may determine the region to be extracted as a secondary image by centering on the bounding box Bx and enlarging each side of the bounding box Bx by a predetermined magnification. The predetermined magnification may be set to, for example, 1.1 times or more and 2.5 times or less, or to 1.5 times or more and 2.0 times or less.

[0079] The method for determining the cutout area described above is illustrative and can be modified as appropriate depending on the types of the first and second equipment. For example, if the first and second equipment are located far apart, the coordinates of the second equipment may be derived based on the coordinate information of the bounding box of the first equipment, and a region centered on the second equipment may be cut out.

[0080] In step S4, the determination unit 74 determines whether or not the fender pole is included in the secondary image (image P2). In other words, step S4 is an example of a determination process according to this disclosure, in which the determination unit 74 inputs the secondary image to the second trained model to determine whether or not the second equipment is included in the secondary image.

[0081] In step S4, the determination unit 74 transmits image P2 to the learning device 80. When the second learning unit 81B of the learning device 80 receives image P2 from the determination unit 74, it inputs image P2 to the second model M2 and transmits the information output from the second model M2 to the information processing unit 70.

[0082] The second model M2 is an inference model that detects and outputs the bounding box By surrounding the fender pole FP from the input image. In this process, the image input to the second model M2 is the image P2 generated by the secondary image generation unit 73.

[0083] "Fender pole" is a class classified by the second model M2. In Embodiment 1, the second model M2 may be an inference model to which only "Fender pole" is applied as a class. The second model M2 may also output a score representing the likelihood of the inference result attached to the bounding box By surrounding the fender pole FP. Such a score may be a numerical value in the range of 0 to 1, corresponding to the degree of confidence that the object indicated by being enclosed by the bounding box By is indeed a fender pole FP.

[0084] Figure 7 shows the bounding box By output by the second model M2 as a result of detecting a fender pole FP when image P2 is input, displayed on the image. The information output from the second model M2 may include a label indicating that the detected object is a fender pole FP and coordinate information of the bounding box By. If no fender pole is detected in image P2, the information output from the second model M2 may include information indicating that image P2 does not contain a fender pole. The determination unit 74 receives the information output from the second model M2.

[0085] The determination unit 74 determines that image P2 includes a fender pole FP if the received information includes a label indicating that it is a fender pole FP. In other words, the determination unit 74 determines that car X is equipped with a fender pole FP. On the other hand, if the received information does not include a label indicating that it is a fender pole FP, or if it includes information indicating that it does not include a fender pole, the determination unit 74 determines that image P2 does not include a fender pole FP. In other words, the determination unit 74 determines that car X is not equipped with a fender pole FP.

[0086] The control unit 7 of the car wash device 2 may control the car wash machine body 4 and wash the automobile X based on the determination result of the determination unit 74. For example, the washing process may be performed while avoiding the fender pole FP.

[0087] Furthermore, in step S4, the identification unit 75 may determine the position of the fender pole FP based on the coordinate information of the bounding box By in the information received from the second model M2. The control unit 7 of the car wash device 2 may control the car wash machine body 4 and wash the car X based on the identification result of the identification unit 75. This can further improve the accuracy of avoidance operations such as washing that avoids the fender pole FP.

[0088] The example described above concerns the case where the second piece of equipment is a fender pole, but the second piece of equipment may also be a side under mirror. Since the side under mirror is also located near the headlight, similar to the fender pole, it can be detected by using a trained model to detect the side under mirror and performing the same processing as for the fender pole.

[0089] In the example described above, primary images were acquired using imaging devices 9R and 9L attached to the car wash machine body 4. However, primary images may also be acquired using imaging device 9G, which is attached to the vehicle height gate 55 and facing in the opposite direction from the car wash machine body 4. By using imaging device 9G, the above process of detecting the fender pole FP before the car wash reception by the remote panel 6 can be performed.

[0090] In the example described above, the control unit 7 and the information processing unit 70 are described as separate components connected in a communicative manner, but this does not limit this embodiment. For example, the control unit 7 and the information processing unit 70 may be physically integrated into a single computer. Furthermore, in the example shown in Figure 1, the learning device 80 is described as an external component of the car wash device 2, but this also does not limit this embodiment. The learning device 80 may be a component of the car wash device 2. In this case, the control unit 7, the information processing unit 70, and the learning device 80 may be integrated into a single computer.

[0091] In the embodiment described above, an example was given in which the first equipment is a headlight and the second equipment is a fender pole, but the combination of the first and second equipment is not limited to this example. For example, the first equipment may be a headlight and the second equipment may be a side under mirror. Alternatively, the first equipment may be a rear window and the second equipment may be a rear wiper.

[0092] The first equipment can preferably be selected from equipment that is essential to an automobile, equipment that produces few false detections when the captured image is input to the detection model, or equipment that is included in all images when the imaging device takes images of multiple automobiles, thus making it easy to increase the amount of training data. The first equipment may be, for example, headlights, taillights, tires, doors, side mirrors, or rear windows.

[0093] Suitable second equipment items may include optional equipment, equipment that frequently produces false positives when captured images are input into the detection model, or equipment that is only included in a portion of the captured images when the imaging device takes images of multiple vehicles, making it difficult to increase the amount of training data. The second equipment items may be, for example, fender poles, side under mirrors, fender mirrors, rear mirrors, spoilers, or decorative items.

[0094] For example, in the above embodiment, if the image captured by the imaging device 9 is directly input into the detection model, the detection accuracy of the fender pole will be low. Furthermore, since the position of the fender pole in the captured image differs depending on the vehicle model, it is difficult to uniformly define the area including the fender pole.

[0095] On the other hand, headlights are equipment that can be detected with high accuracy. Therefore, it is easy to identify the position of the headlights in the captured image. By generating a secondary image from the captured image that includes the fender pole or the position where the fender pole should be installed based on the information of the position identified in this way, and inputting this image into the detection model, the detection accuracy of the fender pole can be significantly improved.

[0096] [Second processing example] A second example of processing by the information processing unit 70 will be explained with reference to Figure 8. Figure 8 is an exemplary flowchart showing the flow of processing performed by the information processing unit 70.

[0097] In step S11, the image acquisition unit 71 acquires the image P1 captured by the imaging device 9 as the primary image. This process is the same as in step S1 of the first processing example.

[0098] In step S12, the detection unit 72 detects the position of the headlight HL in the image P1 acquired by the image acquisition unit 71. This process is the same as in step S2 of the first processing example.

[0099] In step S13, the secondary image generation unit 73 generates a secondary image (image P2) from image P1 that includes the area where the fender pole is located, based on the coordinate information of the bounding box Bx detected by the detection unit 72. Specifically, in step S13, the secondary image generation unit 73 determines the area to be extracted as a secondary image by centering the bounding box Bx and enlarging each side of the bounding box Bx by a predetermined magnification range lower limit.

[0100] In the second processing example, the magnification range, which is the range of the magnification factor of the bounding box Bx used to determine the area to be cut out, may be predetermined. For example, the magnification range may be set to be between 1.1x and 2.5x.

[0101] The upper limit of the magnification range may be determined based on empirical rules. Alternatively, the upper limit of the magnification range may be determined based on the size of the bounding box of a "car" detected using a learning model capable of detecting "cars." Specifically, the upper limit of the magnification range may be determined such that the size of the region magnified by the upper limit of the magnification range is any size between half and one-third of the size of the bounding box of the "car."

[0102] For example, if the magnification range is defined as 1.1x or more and 2.5x or less, in step S13, the secondary image generation unit 73 determines the region obtained by magnifying each side of the bounding box Bx by 1.1x as the region to be extracted as a secondary image, and generates the secondary image.

[0103] In step S14, the determination unit 74 determines whether or not the fender pole is included in the generated secondary image. This process is the same as in step S4 of the first processing example. If the determination in step S14 determines that the fender pole is included in the secondary image (S15: YES), the series of processes is terminated.

[0104] If the determination unit 74 determines that the fender pole is not included in the secondary image (S15: NO), in step S16, the determination unit 74 determines whether the magnification of the secondary image used for the determination was generated exceeds the upper limit. If the magnification exceeds the upper limit (S16: YES), the series of processes is terminated.

[0105] If the magnification does not exceed the upper limit (S16: NO), in step S17, the secondary image generation unit 73 increases the magnification of the bounding box Bx used to determine the region to be cut out, and generates a new secondary image. For example, if the magnification when the secondary image was generated in step S13 was 1.1 times, the magnification when the secondary image is generated in step S17 may be 1.2 times. Each time a new secondary image is generated in step S17, the magnification of the bounding box Bx used to generate the secondary image is increased. Once a new secondary image is generated in step S17, the process returns to step S14.

[0106] [Examples of implementation using software] The function of a car wash system (hereinafter referred to as "the system") can be realized by a program that causes a computer to function as the system, and by a program that causes a computer to function as each control block of the system (particularly each part included in the control unit and the information processing unit).

[0107] In this case, the device includes a computer having at least one control device (e.g., a processor) and at least one storage device (e.g., memory) as hardware for executing the program. By executing the program using this control device and storage device, the functions described in each of the embodiments are realized.

[0108] The above program may be recorded on one or more computer-readable recording media, not temporary ones. These recording media may or may not be provided by the above device. In the latter case, the program may be supplied to the above device via any wired or wireless transmission medium.

[0109] Furthermore, some or all of the functions of each of the above control blocks can also be implemented by logic circuits. For example, an integrated circuit in which logic circuits functioning as each of the above control blocks are formed is also included in the scope of this disclosure. In addition, it is also possible to implement the functions of each of the above control blocks by, for example, a quantum computer.

[0110] 〔summary〕 (1) An information processing device relating to aspect 1 of the present disclosure is an information processing device that determines whether or not a second piece of equipment is installed in an automobile, An image acquisition unit that acquires an image including the aforementioned automobile as a primary image, A detection unit detects the position of a first piece of equipment by inputting the primary image to a first trained model that has been trained to detect a first piece of equipment that is essential equipment for the vehicle, using an image including the vehicle as input information. A secondary image generation unit generates a secondary image from the primary image, based on position information indicating the position of the first equipment detected by the detection unit, which includes the region where the second equipment would be located if the vehicle were equipped with the second equipment. The system includes a determination unit that determines whether or not the second equipment is included in the secondary image by inputting the secondary image to a second trained model that has been trained to detect the second equipment provided by the automobile, using an image including the automobile as input information.

[0111] This configuration improves the detection accuracy of the second piece of equipment.

[0112] (2) In the information processing device according to Embodiment 2 of the present disclosure, in Embodiment 1, the detection unit generates a bounding box surrounding the first equipment in the primary image and detects the position of the first equipment in the primary image based on the coordinates of the bounding box.

[0113] (3) The information processing device according to Embodiment 3 of the present disclosure further includes, in Embodiment 1 or 2 above, a unit that identifies the position of the second equipment by inputting the secondary image to the second trained model.

[0114] (4) In any of the above embodiments 1 to 3, the information processing device according to embodiment 4 of the present disclosure is an optional piece of equipment.

[0115] (5) In any of the embodiments 1 to 4 above, the information processing device according to embodiment 5 of the present disclosure is characterized in that the first equipment is a headlight and the second equipment is a fender pole or a side under mirror.

[0116] (6) In any of the embodiments 1 to 5 described above, the information processing device according to embodiment 6 of the present disclosure is an image obtained by cropping the primary image.

[0117] (7) The determination program according to Embodiment 7 of the present disclosure is a determination program for causing a computer to function as an information processing device according to Embodiment 1, and is a determination program for causing the computer to function as the image acquisition unit, the detection unit, the secondary image generation unit, and the determination unit.

[0118] (8) The determination method relating to aspect 8 of this disclosure is: An image acquisition process that acquires an image including a car as the primary image, A detection step in which the position of a first piece of equipment is detected by inputting the primary image into a first trained model that has been trained to detect a first piece of equipment installed on a vehicle, using an image including the vehicle as input information, A secondary image generation step, based on position information indicating the position of the first equipment detected in the detection step, generates a secondary image from the primary image that includes the region where the second equipment would be located if the vehicle were equipped with the second equipment. The method includes a determination step of determining whether or not the second equipment is included in the secondary image by inputting the secondary image into a second trained model that has been trained to detect the second equipment provided by the automobile, using an image including the automobile as input information.

[0119] (9) The car wash device according to aspect 9 of the present disclosure is The car wash machine itself, and An imaging unit that captures an image including the aforementioned automobile, An information processing device according to any one of claims 1 to 6, The system includes a control unit that controls the operation of the car wash machine body based on the determination result of the determination unit.

[0120] [Additional Notes] This disclosure is not limited to the embodiments described above, and various modifications are possible within the scope of the claims. Embodiments obtained by appropriately combining the technical means disclosed in different embodiments are also included in the technical scope of this disclosure. [Explanation of Symbols]

[0121] 100... Car Wash System 2...Car wash equipment 7. Control Unit 9, 9L, 9R... Imaging devices 70. Information Processing Unit (Information Processing Equipment) 71...Image acquisition unit 72.. Detection Unit 73...Secondary Image Generation Unit 74...judgment section 75...Specific part 80...Learning device 81... Control Unit 81A...1st Learning Department 81B...Second Learning Department 82...Storage section M1 First Model (First Trained Model) M2 Second Model (Second Trained Model)

Claims

1. An information processing device for determining whether or not a second piece of equipment is installed in an automobile, An image acquisition unit that acquires an image including the aforementioned automobile as a primary image, A detection unit detects the position of a first piece of equipment by inputting the primary image into a first trained model that has been trained to detect a first piece of equipment that is essential equipment for the vehicle, using an image including the vehicle as input information. A secondary image generation unit generates a secondary image from the primary image, based on position information indicating the position of the first equipment detected by the detection unit, which includes the region where the second equipment would be located if the vehicle were equipped with the second equipment. An information processing device comprising: a determination unit that determines whether or not a second piece of equipment is included in a secondary image by inputting the secondary image to a second trained model that has been trained to detect a second piece of equipment provided by a vehicle, using an image including the vehicle as input information.

2. The information processing apparatus according to claim 1, wherein the detection unit generates a bounding box surrounding the first equipment in the primary image and detects the position of the first equipment in the primary image based on the coordinates of the bounding box.

3. The information processing apparatus according to claim 1, wherein the second piece of equipment is an optional piece of equipment.

4. The information processing device according to claim 1, wherein the first piece of equipment is a headlight, and the second piece of equipment is a fender pole or a side under mirror.

5. The information processing apparatus according to claim 1, wherein the secondary image is an image obtained by cropping the primary image.

6. A determination program for causing a computer to function as an information processing device according to claim 1, wherein the image acquisition unit, the detection unit, the secondary image generation unit, and the determination unit are the determination program for causing the computer to function.

7. An image acquisition process that acquires an image including a car as the primary image, A detection step in which the position of a first piece of equipment is detected by inputting the primary image into a first trained model that has been trained to detect a first piece of equipment installed on a vehicle, using an image including the vehicle as input information, A secondary image generation step, based on position information indicating the position of the first equipment detected in the detection step, generates a secondary image from the primary image that includes the region where the second equipment would be located if the vehicle were equipped with the second equipment. A determination method comprising: a determination step of determining whether or not the second equipment is included in the secondary image by inputting the secondary image to a second trained model that has been trained to detect the second equipment provided by the automobile, using an image including the automobile as input information.

8. The car wash machine itself, and An imaging device that captures an image including the aforementioned automobile, An information processing device according to any one of claims 1 to 5, A car wash device comprising: a control unit that controls the operation of the car wash machine body based on the determination result of the determination unit.

Images