Information processing apparatus

Abstract

An information processing device is provided with a processor configured to execute: a process for acquiring a peripheral image in which a road mirror has been captured, the peripheral image being generated by imaging the periphery of a first vehicle by an imaging unit mounted on the first vehicle; when it is determined that at least one of a second vehicle and a pedestrian approaching the installation location from a direction different from a direction in which the first vehicle approaches the installation location of the road mirror is captured in the mirror surface of the road mirror captured in the acquired peripheral image, it is determined that there is no abnormality in the orientation of the road mirror.

Description

Technical Field

[0001] The present disclosure relates to an information processing device. Background Art

[0002] A camera unit is provided on the vehicle to capture the surroundings of the vehicle. When the vehicle is traveling on a road, a road reflector installed on the road may be captured in an image generated by the camera unit, i.e., a peripheral image. Japanese Patent Publication No. 2020-113070 discloses a technology for determining a road reflector whose visual confirmation has deteriorated based on a peripheral image of the road reflector and correct answer data indicating that the road reflector is installed at a normal position, height, and angle.

[0003] In the technology disclosed in the above patent document, if there is no correct answer data for the case where the road reflector is installed at a normal position, height, and angle, it is impossible to determine whether the direction of the road reflector is abnormal. Summary of the invention

[0004] According to one aspect of the present disclosure, an information processing device is provided. The information processing device includes a processor, and the processor is configured to perform the following processing, namely: obtaining a peripheral image of a road reflector installed on a road on which the first vehicle is traveling, which is generated by a camera unit mounted on a first vehicle photographing the periphery of the first vehicle; when it is determined that at least one of a second vehicle and a pedestrian approaching the installation location of the road reflector from a direction different from the direction in which the first vehicle approaches the installation location of the road reflector is captured in the mirror surface of the road reflector captured in the acquired peripheral image, it is determined that there is no abnormality in the direction of the road reflector. BRIEF DESCRIPTION OF THE DRAWINGS

[0005] Figure 1 A diagram for explaining an information processing system.

[0006] Figure 2 A diagram showing an example of the configuration of a server device and a control device.

[0007] Figure 3 This is a diagram showing a portion of an enlarged image of the periphery of a road reflector that captures a normal orientation.

[0008] Figure 4 This is a diagram showing a portion of an enlarged image of the periphery of a road reflector that captures an abnormal orientation.

[0009] Figure 5 The following is a flowchart showing an example of processing executed by the control device.

[0010] Figure 6 This is a flowchart showing an example of processing executed by the server device. DETAILED DESCRIPTION

[0011] (Implementation Method)

[0012] Below, according to Figures 1 to 6 Next, one embodiment of the information processing device will be described.

[0013] (Overall Structure of Information Processing System 1)

[0014] like Figure 1 As shown, the information processing system 1 includes, for example, one or more vehicles 10 and one or more server devices 20. The server device 20 is an example of an information processing device. Specifically, the server device 20 executes a process for determining whether the orientation of a road reflector 50 installed on a road is abnormal.

[0015] (Road reflector 50)

[0016] The road reflector 50 is set at a position where the field of vision on the road is poor. For example, the road reflector 50 is set at an intersection or a turn. On the road, the place where the road reflector 50 is set is sometimes referred to as the "setting location 60". The road reflector 50 assists the driver of the vehicle 10 approaching the setting location 60 from a first direction to confirm whether there are other vehicles or pedestrians approaching the setting location 60 from a second direction different from the first direction. In addition, the road reflector 50 assists the driver of the vehicle 10 approaching the setting location 60 from the second direction to confirm whether there are other vehicles or pedestrians approaching the setting location 60 from the first direction.

[0017] The first direction refers to, for example, the left side in the width direction of the mirror surface when facing the mirror surface of the road reflector 50. The second direction refers to, for example, the right side in the width direction of the mirror surface when facing the mirror surface of the road reflector 50. And, the vehicle 10 approaching the installation place 60 from the first direction is an example of the first vehicle. In addition, the vehicle 10 approaching the installation place 60 from the second direction is an example of the second vehicle. In addition, the first direction and the second direction are examples, and are not limited thereto, and the left-right relationship can also be reversed.

[0018] exist Figure 1In the example shown, the installation location 60 of the road reflector 50 is an intersection where a first road 61 and a second road 62 intersect. In this case, the road reflector 50 is used to allow the driver of the vehicle 10 approaching the installation location 60 from the first road 61 to confirm whether there are other vehicles or pedestrians approaching the installation location 60 from the second road 62. In addition, the road reflector 50 is used to allow the driver of the vehicle 10 approaching the installation location 60 from the second road 62 to confirm whether there are other vehicles or pedestrians approaching the installation location 60 from the first road 61.

[0019] exist Figure 1 In the example shown, the direction approaching the installation site 60 of the road reflector 50 from the first road 61 corresponds to the first direction. In addition, the direction approaching the installation site 60 from the second road 62 corresponds to the second direction.

[0020] (Structure of Vehicle 10)

[0021] The vehicle 10 includes, for example, an imaging unit 11 , a position information acquisition unit 12 , and a control device 30 .

[0022] The camera unit 11 is, for example, a digital camera that utilizes a solid-state imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal Oxide Semiconductor). An example of the camera unit 11 is a digital camera that constitutes a driving recorder. The camera unit 11 captures the surrounding conditions of the vehicle 10. Depending on the installation position of the camera unit 11 in the vehicle 10, the camera unit 11 can capture the front, rear, and sides of the vehicle 10. The camera unit 11 generates a surrounding image by capturing the surrounding conditions of the vehicle 10. When the vehicle 10 approaches the installation location 60 of the road reflector 50, the road reflector 50 may sometimes be captured in the surrounding image generated by the camera unit 11.

[0023] The position information acquisition unit 12 is, for example, a GNSS (Global Navigation Satellite System) receiver. The position information acquisition unit 12 identifies the position of the vehicle 10 based on a signal received from a GNSS satellite, and acquires position information indicating the position of the vehicle 10 .

[0024] The control device 30 and the server device 20 are connected to each other via a communication network 40 in a manner capable of sending and receiving information. The communication network 40 includes, for example, the Internet, a WAN (Wide Area Network), a LAN (Local Area Network), a provider terminal, a wireless communication network, a wireless base station, a dedicated line, etc. When the road reflector 50 is captured in the surrounding image generated by the camera unit 11, the control device 30 sends the surrounding image to the server device 20.

[0025] (Structure of Control Device 30)

[0026] like Figure 2 As shown, the control device 30 includes, for example, a processing circuit 31, a communication unit 32, and a storage unit 33. The processing circuit 31 is composed of, for example, a hardware processor such as a CPU (Central Processing Unit). The hardware processor enables the processing circuit 31 to perform various processes by executing a program. The program may also be pre-stored in the storage unit 33. The storage unit 33 is implemented by, for example, a non-temporary storage medium such as a HDD (Hard Disk Drive) and a flash memory. The communication unit 32 includes, for example, a communication circuit or a communication module, and sends and receives various data with the server device 20 and other devices based on the control of the processing circuit 31.

[0027] The storage unit 33 stores, for example, a learned model 331. The learned model 331 is a model that performs machine learning to determine whether the road reflector 50 is captured in the surrounding image generated by the camera unit 11. An example of the learned model 331 is a neural network. When the surrounding image is input as an input variable to the learned model 331, the learned model 331 outputs an output variable indicating the probability of capturing the road reflector 50 in the surrounding image. For example, the higher the probability of capturing the road reflector 50 in the surrounding image, the larger the value output by the learned model 331 as the output variable. In this case, the processing circuit 31 of the control device 30 determines that the road reflector 50 is captured in the surrounding image when the output variable is greater than the threshold. On the other hand, the processing circuit 31 determines that the road reflector 50 is not captured in the surrounding image when the output variable is less than the threshold. The processing circuit 31 causes the communication unit 32 to send the surrounding image determined to capture the road reflector 50 and the position information of the position where the surrounding image is captured to the server device 20.

[0028] (Structure of Server Device 20)

[0029] like Figure 2As shown, the server device 20 includes, for example, a processing circuit 21, a communication unit 22, and a storage unit 23. The processing circuit 21 is composed of, for example, a hardware processor such as a CPU. The hardware processor can cause the processing circuit 21 to perform various processes by executing a program. The program can also be pre-stored in the storage unit 23. The storage unit 23 is implemented by, for example, a non-temporary storage medium such as an HDD or a flash memory. The communication unit 22 includes, for example, a communication circuit or a communication module, and sends and receives various data with the control device 30 and other devices based on the control of the processing circuit 21.

[0030] The storage unit 23 stores, for example, a learned model 231 and road reflector information 232. The learned model 231 is a machine learning model that performs instance segmentation of objects captured on the mirror surface of the road reflector 50. An example of the learned model 231 is a convolutional neural network (CNN). When a surrounding image is input as an input variable to the learned model 231, the learned model 231 outputs a segmentation result representing each object captured on the mirror surface within the range of pixels constituting the mirror surface of the road reflector 50 in the surrounding image. In the segmentation, for example, other vehicles, pedestrians, bicycles, roadside trees, roads, sidewalks, traffic lights, dividing lines, road signs, and buildings that may exist around the road reflector 50 are identified.

[0031] Next, in the instance segmentation of the learned model 231, by identifying objects captured in the mirror surface of the road reflector 50 and objects captured outside the mirror surface of the road reflector 50 in the peripheral image, it is possible to identify whether the vehicle captured in the mirror surface of the road reflector 50 is the own vehicle or another vehicle. The own vehicle refers to a vehicle 10 equipped with a camera unit 11 that generates raw data of the peripheral image as a segmentation result.

[0032] Specifically, when the mirror surface of the road reflector 50 captured in the peripheral image faces the front, the learned model 231 recognizes that the vehicle captured in the mirror surface of the road reflector 50 is the vehicle itself. When the road reflector 50 is tilted from the normal orientation, the mirror surface of the road reflector 50 faces the vehicle 10 equipped with the camera unit 11. Therefore, the mirror surface of the road reflector 50 facing the front is captured in the peripheral image. On the other hand, when the mirror surface of the road reflector 50 captured in the peripheral image does not face the front, the learned model 231 recognizes that the vehicle captured in the mirror surface of the road reflector 50 is another vehicle. When the road reflector 50 is in the normal orientation, the mirror surface of the road reflector 50 does not face the vehicle 10 equipped with the camera unit 11. Therefore, in the peripheral image, the mirror surface of the road reflector 50 that is not facing the front is captured. Specifically, in the peripheral image, the road reflector 50 whose mirror surface faces the center of the first direction and the second direction is captured. The mirror surface of the road reflector 50 is substantially circular. The learned model 231 distinguishes the own vehicle from other vehicles based on whether the mirror surface of the road reflector 50 is a perfect circle or an ellipse, for example. The details of the road reflector information 232 will be described later.

[0033] When receiving the surrounding image and the position information from the control device 30, the processing circuit 21 inputs the surrounding image into the learned model 231. Furthermore, the processing circuit 21 obtains the segmentation result as the output of the learned model 231. Based on the segmentation result, the processing circuit 21 determines whether at least one of another vehicle and a pedestrian is captured on the mirror surface of the road reflector 50.

[0034] When the segmentation result includes at least one of the other vehicles and pedestrians, the processing circuit 21 determines that at least one of the other vehicles and pedestrians is captured on the mirror surface of the road reflector 50. When the processing circuit 21 determines that at least one of the other vehicles and pedestrians is captured on the mirror surface of the road reflector 50, it determines that there is no abnormality in the direction of the road reflector 50.

[0035] exist Figure 3 as well as Figure 4 In FIG. 5 , a peripheral image captured and generated by the camera unit 11 of the vehicle 10 traveling on the first road 61 is shown with the range 80 of the mirror surface of the road reflector 50 enlarged. Figure 3As shown, when the direction of the road reflector 50 is normal, the second road 62, other vehicles 91 traveling on the second road 62, and pedestrians 92 walking on the second road 62 are photographed in the range 80 of the mirror surface. Therefore, the segmentation result includes other vehicles, pedestrians, and roads. Therefore, the processing circuit 21 determines based on the segmentation result that at least one of the other vehicles 91 and the pedestrian 92 is photographed in the mirror surface of the road reflector 50, and determines that the direction of the road reflector 50 is normal.

[0036] like Figure 4 As shown, in the case where the direction of the road reflector 50 is abnormal, the second road 62, other vehicles 91 traveling on the second road 62, and pedestrians 92 walking on the second road 62 will not be photographed within the scope 80 of the mirror surface. On the other hand, depending on the direction of the road reflector 50, only the sky or the roadside trees will be photographed. In addition, the scope 80 of the mirror surface in the case of abnormal direction of the road reflector 50 is an example and is not limited to this. There are various ways, such as either photographing the vehicle 10 as the vehicle traveling on the first road 61 and the first road 61 within the scope 80 of the mirror surface, or only photographing the ground. In either case, the segmentation result in the case of abnormal direction of the road reflector 50 does not include other vehicles and pedestrians. Therefore, the processing circuit 21 determines that other vehicles 91 and pedestrians 92 are not photographed in the mirror surface of the road reflector 50 based on the segmentation result.

[0037] On the other hand, even if the direction of the road reflector 50 is normal, when there are no other vehicles 91 and pedestrians 92 on the second road 62, the other vehicles 91 and pedestrians 92 will not be photographed in the range 80 of the mirror surface. In this regard, when the processing circuit 21 determines that the other vehicles 91 and pedestrians 92 are not photographed in the mirror surface of the road reflector 50, it derives the proportion of the portion of the road photographed in the range 80 of the mirror surface of the road reflector 50 captured in the peripheral image based on the segmentation result. The processing circuit 21 determines whether the derived proportion is greater than a predetermined proportion.

[0038] When the direction of the road reflector 50 is abnormal, the road is not captured at all or the above ratio is very small even if the road is captured in the mirror surface of the road reflector 50. In such a case, even if there are vehicles or pedestrians passing on the road, the vehicles or pedestrians may not be captured in the mirror surface of the road reflector 50. Therefore, for example, a predetermined ratio is set as a criterion for determining whether a vehicle or pedestrian passing on the road can be captured in the mirror surface.

[0039] The predetermined ratio is set in advance based on, for example, a surrounding image that satisfies conditions (A1) and (A2).

[0040] (A1) The imaging unit 11 captures and generates a scene in which the direction of the road mirror 50 is normal.

[0041] (A2) There are no other vehicles 91 or pedestrians 92 in the second direction.

[0042] Specifically, the predetermined ratio is the ratio of road pixels to pixels in the range 80 of the mirror surface of the road reflector 50 captured in the surrounding image that satisfies the conditions (A1) and (A2). The predetermined ratio is, for example, a statistical value of the ratio calculated based on surrounding images at various positions and a plurality of surrounding images that satisfy the conditions (A1) and (A2).

[0043] If the derived ratio is greater than the predetermined ratio, the processing circuit 21 determines that there is no abnormality in the orientation of the road reflector 50. On the other hand, if the derived ratio is less than the predetermined ratio, the processing circuit 21 determines that there is an abnormality in the orientation of the road reflector 50.

[0044] The processing circuit 21 updates the road reflector information 232 based on the judgment result. The road reflector information 232 includes one or more records in which identification information that can identify the road reflector 50, information indicating the direction of approaching the installation location 60 of the road reflector 50, and information indicating the judgment result of the direction of the road reflector 50 are mutually associated. The identification information that can identify the road reflector 50 is, for example, information indicating the installation location 60 of the road reflector 50.

[0045] The processing circuit 21 searches the road reflector information 232 by using the position information obtained from the control device 30 as a search keyword, thereby determining the record related to the road reflector 50 of the judgment object. Specifically, the processing circuit 21 determines the record of the road reflector 50 having the identification information indicating that it is within a predetermined distance from the position information as the record of the road reflector 50 of the judgment object. In addition, the processing circuit 21 determines the direction of approaching the installation location 60 of the road reflector 50 for the vehicle 10 that sent the position information based on the positional relationship between the position information and the identification information. As described above, the direction of approaching the installation location 60 of the road reflector 50 is the first direction or the second direction. Based on the determined approaching direction and the judgment result of the direction of the road reflector 50, the processing circuit 21 generates or updates the record of the road reflector 50 of the judgment object, thereby updating the road reflector information 232.

[0046] Based on the updated road reflector information 232 , the processing circuit 21 determines that the orientation of the road reflector 50 is normal for a road reflector 50 for which both conditions ( B1 ) and ( B2 ) are satisfied.

[0047] (B1) Based on the surrounding image generated by the imaging unit 11 of the first vehicle approaching the installation location 60 of the road reflector 50 from the first direction, it is determined that there is no abnormality in the orientation of the road reflector 50 .

[0048] (B2) Based on the surrounding image generated by the imaging unit 11 of the second vehicle approaching the installation location 60 of the road reflector 50 from the second direction, it is determined that there is no abnormality in the orientation of the road reflector 50 .

[0049] The processing circuit 21 determines whether both conditions (B1) and (B2) are satisfied for each record of the updated road reflector information 232. Furthermore, the processing circuit 21 determines that the orientation of the road reflector 50 is normal for the road reflector 50 for which both conditions are satisfied.

[0050] (Processing by the control device 30)

[0051] Reference Figure 5 Next, a series of processes executed by the control device 30 in order to transmit the peripheral image captured by the road reflector 50 to the server device 20 will be described. This series of processes is executed by the processing circuit 31 at every predetermined control cycle.

[0052] The processing circuit 31 obtains the surrounding image generated by the camera unit 11 (step S100). The processing circuit 31 determines whether the road reflector 50 is captured in the obtained surrounding image (step S102). When the processing circuit 31 determines that the road reflector 50 is not captured in the surrounding image (step S102: No), since there is no information to be sent to the server device 20, a series of processes are terminated. When the processing circuit 31 determines that the road reflector 50 is captured in the surrounding image (step S102: Yes), the processing circuit 31 obtains the position information from the position information acquisition unit 12 (step S104). The position information obtained by the processing circuit 31 in step S104 is the position information of the vehicle 10 when the surrounding image obtained in step S100 is generated by the camera unit 11. The processing circuit 31 causes the acquired surrounding image and position information to be sent from the communication unit 32 to the server device 20 (step S106). Then, the processing circuit 31 ends a series of processes.

[0053] (Processing of Server Device 20)

[0054] Reference Figure 6 , a series of processes performed by the server device 20 to determine whether the direction of the road reflector 50 is normal based on the surrounding image received from the vehicle 10 will be described. When the communication unit 22 receives the surrounding image from the vehicle 10, the processing circuit 21 of the server device 20 performs the series of processes.

[0055] The processing circuit 21 obtains the surrounding image and position information sent from the control device 30 of the vehicle 10 (step S200). The processing circuit 21 inputs the obtained surrounding image into the learned model 231, thereby obtaining the segmentation result as the output of the learned model 231 (step S202). Based on the segmentation result, the processing circuit 21 determines whether at least one of the other vehicles 91 and the pedestrian 92 is captured in the mirror surface of the road reflector 50 captured in the surrounding image (step S204). When the segmentation result includes at least one of the other vehicles and the pedestrian, the processing circuit 21 determines that at least one of the other vehicles 91 and the pedestrian 92 is captured in the mirror surface of the road reflector 50 captured in the surrounding image. In addition, when the other vehicles and the pedestrian are not included in the segmentation result, the processing circuit 21 determines that the other vehicles 91 and the pedestrian 92 are not captured in the mirror surface of the road reflector 50 captured in the surrounding image.

[0056] When the processing circuit 21 determines that at least one of the other vehicles 91 and the pedestrian 92 is captured on the mirror surface of the road reflector 50 captured in the peripheral image (step S204: Yes), it is determined that there is no abnormality in the orientation of the road reflector 50 (step S206). When the processing circuit 21 determines that no other vehicles 91 and the pedestrian 92 are captured on the mirror surface of the road reflector 50 captured in the peripheral image (step S204: No), it derives the proportion of the portion of the road captured within the range 80 of the mirror surface of the road reflector 50 captured in the peripheral image (step S208). The processing circuit 21 determines whether the derived proportion is greater than a predetermined proportion (step S210). When the derived proportion is greater than a predetermined proportion (step S210: Yes), the processing circuit 21 determines that there is no abnormality in the orientation of the road reflector 50 (step S206). When the derived ratio is smaller than the predetermined ratio (step S210 : No), the processing circuit 21 determines that the orientation of the road reflector 50 is abnormal (step S212 ).

[0057] The processing circuit 21 updates the road reflector information 232 based on the judgment result of step S206 or step S212 (step S214). Next, the processing circuit 21 judges whether there is no abnormality when observing from both the first direction and the second direction based on the updated road reflector information 232 (step S216). When both conditions (B1) and (B2) are met, the processing circuit 21 judges that there is no abnormality in both directions (step S216: yes). For the road reflector 50 for which both conditions (B1) and (B2) are met, the processing circuit 21 judges that the orientation of the road reflector 50 is normal (step S218). And, the processing circuit 21 ends a series of processing. In addition, when both conditions (B1) and (B2) are not met (step S216: no), the processing circuit ends a series of processing.

[0058] (Functions and Effects of Implementation Methods)

[0059] According to the above-described embodiment, the following operations and effects can be obtained.

[0060] (1) When the vehicle 10 is traveling, the camera unit 11 generates a surrounding image by photographing the surroundings of the vehicle 10. When the vehicle 10 approaches the installation location 60 of the road reflector 50, the road reflector 50 may be captured in the surrounding image generated by the camera unit 11 of the vehicle 10. The control device 30 acquires the surrounding image acquired from the camera unit 11. At this time, when the control device 30 determines that the road reflector 50 is captured in the acquired surrounding image, the control device 30 transmits the surrounding image to the server device 20.

[0061] The processing circuit 21 of the server device 20 obtains the surrounding image sent from the control device 30. In addition, the processing circuit 21 determines whether at least one of the other vehicles 91 and the pedestrian 92 is captured in the mirror surface of the road reflector 50 captured in the obtained surrounding image. When the processing circuit 21 determines that at least one of the other vehicles 91 and the pedestrian 92 is captured in the mirror surface of the road reflector 50 captured in the obtained surrounding image, it determines that there is no abnormality in the orientation of the road reflector 50. According to such a structure, the server device 20 can determine whether the orientation of the road reflector 50 is abnormal by using the surrounding image generated by the camera unit 11 without using the correct answer data when the road reflector 50 is installed at a normal position, height, and angle.

[0062] (2) When the vehicle 10 approaches the installation location 60 of the road reflector 50 from the first direction, there may be no other vehicles or pedestrians approaching the installation location 60 from the second direction. In this case, even if the orientation of the road reflector 50 is normal, other vehicles or pedestrians will obviously not be captured on the mirror surface of the road reflector 50. When the orientation of the road reflector 50 is normal, the road connected to the installation location 60 should be captured on the mirror surface of the road reflector 50. In other words, when the road cannot be captured on the mirror surface of the road reflector 50, or when the road is captured on the mirror surface but most of the mirror surface captures scenery other than the road, it can be considered that the orientation of the road reflector 50 is abnormal.

[0063] Then, the processing circuit 21 derives the proportion of the portion of the road captured within the range 80 of the mirror surface of the road reflector 50 captured in the surrounding image. And, when the derived proportion is greater than a predetermined proportion, the processing circuit 21 determines that there is no abnormality in the orientation of the road reflector 50. According to such a configuration, the server device 20 can determine whether the orientation of the road reflector 50 is abnormal even when there are no other vehicles 91 or pedestrians 92 around the installation location 60 of the road reflector 50.

[0064] (3) The processing circuit 21 refers to the road reflector information 232, and when both conditions (B1) and (B2) are satisfied, it is determined that the orientation of the road reflector 50 is normal. Here, sometimes, although the road reflector 50 is in a normal orientation when viewed from the first direction, it is in an abnormal orientation when viewed from the second direction. In other words, it can be said that the orientation of the road reflector 50 is normal only when both the first direction and the second direction are not abnormal. According to such a structure, the server device 20 can determine whether the orientation of the road reflector 50 is normal based on the road reflector information 232.

[0065] (Change Example)

[0066] The above-mentioned embodiment may also be modified as follows. In addition, the above-mentioned embodiment and the following individual examples may be combined with each other within the scope of no technical contradiction.

[0067] The camera unit 11 can generate a video composed of a plurality of surrounding images that are continuous in time sequence. Therefore, the processing circuit 31 may also send a video based on the surrounding image determined to have captured the road reflector 50 to the server device 20. In this case, the processing circuit 31 generates a video composed of surrounding images that are continuously captured and generated in time sequence from a predetermined time before to a predetermined time after the timing when the surrounding image determined to have captured the road reflector 50 is captured and generated. Figure 5In step S106 in the process, the processing circuit 31 sends the generated video together with the position information to the server device 20.

[0068] The processing circuit 21 of the server device 20 determines whether at least one of the other vehicles 91 and the pedestrian 92 is captured in the mirror surface of the road reflector 50 captured in the video based on the acquired video. Specifically, when the video satisfies condition (C), the processing circuit 21 determines that at least one of the other vehicles 91 and the pedestrian 92 is captured in the mirror surface of the road reflector 50 captured in the surrounding image.

[0069] (C) The duration of the state in which at least one of the other vehicle 91 and the pedestrian 92 is captured on the mirror surface of the road reflector 50 is longer than a predetermined time.

[0070] exist Figure 6 In step S202, the processing circuit 21 inputs each peripheral image constituting the video as an input variable to the learned model 231. The processing circuit 21 obtains the segmentation results as outputs of the learned model 231. In the following description, a plurality of segmentation results corresponding to peripheral images that are continuously captured and generated in chronological order are also described as continuous segmentation results.

[0071] The processing circuit 21 determines whether at least one of other vehicles and pedestrians is included in the mirror surface of the road reflector 50 for each continuous segmentation result. The processing circuit 21 determines that the condition (C) is satisfied when at least one of other vehicles and pedestrians is included in all continuous segmentation results. In addition, the processing circuit 21 determines that the condition (C) is not satisfied when no other vehicles and pedestrians are included in the continuous segmentation results. In addition, the processing circuit 21 can also determine that the condition (C) is satisfied when at least one of other vehicles and pedestrians is continuously included in a predetermined number of segmentation results in the continuous segmentation results based on the relationship between the frame rate of the video and the predetermined time. In this case, the predetermined number is the product of the frame rate and the predetermined time.

[0072] Here, if the orientation of the road reflector 50 is normal, other vehicles 91 or pedestrians 92 approaching the installation location 60 of the road reflector 50 from a distance should be continuously captured on the mirror surface of the road reflector 50. When the video satisfies condition (C), the processing circuit 21 determines that at least one of the other vehicles 91 and the pedestrian 92 is captured on the mirror surface of the road reflector 50 captured in the peripheral image (step S204: yes). Furthermore, when the video satisfies condition (C), the processing circuit 21 determines that there is no abnormality in the orientation of the road reflector 50 (step S206). Therefore, the server device 20 can determine whether the orientation of the road reflector 50 is abnormal based on the video.

[0073] The processing circuit 21 may not perform the processing of steps S214 to S218. Figure 6 In addition to the series of processes shown, the processes of steps S214 to S218 are executed as a series of processes at predetermined time intervals. In this case, the predetermined time is a time for which the determination results of the road reflection mirror 50 are sufficiently collected.

[0074] The processing circuit 21 may also output part or all of the judgment results obtained in step S206, step S212, and step S218 to an information terminal of a worker who manages the road. Thus, the worker can use the judgment results related to the orientation of the road reflector 50 for maintenance management of the road reflector 50.

[0075] The processing circuit 31 may determine whether the road reflector 50 is captured in the peripheral image based on the result obtained by general image analysis instead of or in addition to the process of determining whether the road reflector 50 is captured in the peripheral image using the learned model 331 .

[0076] Instead of or in addition to the process of obtaining the segmentation result using the learned model 231 , the processing circuit 21 may determine whether at least one of the other vehicle 91 and the pedestrian 92 is captured on the mirror surface of the road reflector 50 based on the result obtained by general image analysis.

[0077] The processing circuit 31 may send the surrounding image only when the server device 20 requests the server device 20 to provide the surrounding image. In this case, the processing circuit 31 sends the position information to the server device 20 at predetermined time intervals. The server device 20 requests the vehicle 10 to provide the surrounding image when the vehicle 10 is near the road reflector 50 based on the received position information. The processing circuit 31 sends the surrounding image generated at the timing of the request to the server device 20.

[0078] The processing circuit 21 of the server device 20 may also perform a part of the various judgments performed by the processing circuit 31. In this case, each time the camera unit 11 generates a surrounding image, the processing circuit 31 associates the surrounding image with the position information obtained by the position information acquisition unit 12 when the surrounding image is generated by the camera unit 11, and transmits the associated image to the server device 20. Based on the surrounding image received from the control device 30, the processing circuit 21 determines whether the road reflector 50 is captured in the surrounding image. If the processing circuit 21 determines that the road reflector 50 is captured in the surrounding image, it executes Figure 6The series of processes shown are the processes after step S202.

[0079] The control device 30 of the vehicle 10 may also execute the equivalent Figure 6 In this case, the control device 30 determines whether the direction of the road reflector 50 is abnormal. That is, the control device 30 functions as an information processing device.

[0080] Some or all of the components of the processing circuit 21 and the processing circuit 31 may be implemented by hardware such as LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), GPU (Graphics Processing Unit), or by collaboration between software and hardware.

Claims

1. An information processing device comprising a processor, The processor is configured to perform the following processing, namely: acquiring a peripheral image of a road reflector provided on a road on which the first vehicle is traveling, from a peripheral image generated by photographing the periphery of the first vehicle by a camera unit mounted on the first vehicle; When it is determined that a second vehicle and at least one of a pedestrian approaching the installation location of the road reflector from a direction different from the direction in which the first vehicle approaches the installation location of the road reflector are captured on the mirror surface of the road reflector captured in the acquired surrounding image, it is determined that there is no abnormality in the orientation of the road reflector.

2. The information processing device according to claim 1, wherein: The surrounding image is one of a plurality of surrounding images that are continuous in time sequence. The processor is configured to execute the following processing, namely, obtaining a video composed of the plurality of surrounding images, and When it is determined that the duration of the state in which at least one of the second vehicle and the pedestrian is captured on the mirror surface of the road reflector captured in the video is longer than a predetermined time, it is determined that at least one of the second vehicle and the pedestrian is captured on the mirror surface of the road reflector captured in the surrounding image.

3. The information processing device according to claim 1, wherein: The processor is configured to perform the following processing, namely: deriving a ratio of a portion of the road captured within a range of the mirror surface of the road reflector captured in the peripheral image; When the derived ratio is equal to or greater than a predetermined ratio, it is determined that there is no abnormality in the orientation of the road reflector.

4. The information processing device according to claim 1, wherein: The processor is configured to perform the following processing, namely, When both of the following conditions are met, it is determined that the orientation of the road reflector is normal: determining that there is no abnormality in the orientation of the road reflector based on the surrounding image generated by the camera unit of the first vehicle approaching the installation location of the road reflector from a first direction and capturing the surroundings of the first vehicle; Based on the surrounding image generated by the imaging unit of the second vehicle approaching the installation location from a second direction different from the first direction and capturing the surroundings of the second vehicle, it is determined that there is no abnormality in the orientation of the road reflector.

Images