Warning device, system and method, and program

Abstract

A warning device (1) comprises: a region recognition unit (11) that recognizes a region included in an image which has captured the surroundings of a travelling vehicle; a first determination unit (12) that determines the necessity of character recognition, on the basis of the type of that recognized region; a character recognition unit (13) that performs character recognition on the recognized region if character recognition was determined to be necessary; a second determination unit (14) that determines the necessity of a warning, on the basis of the results of the character recognition and vehicle specification information; and an output unit (15) that outputs warning information if a warning was determined to be necessary.

Description

Alert device, system and method, and computer-readable medium

[0001] The present disclosure relates to a warning device, a system, a method, and a program.

[0002] Patent Literature 1 discloses a technology relating to a driving assistance device that detects vehicle traffic regulation information from image information provided by a front camera of the vehicle, generates notification information based on vehicle specification information and the traffic regulation information, and notifies the driver of the vehicle of the notification information. Patent Literature 2 also discloses a technology relating to a vehicle warning device. The vehicle warning device of Patent Literature 2 measures the height of an elevated structure on a road by irradiating a laser beam from a traveling vehicle, determines whether the vehicle can pass under the elevated structure based on the measurement result, and issues a warning based on the determination result.

[0003] JP 2014-006707 A JP 2021-077201 A

[0004] Therefore, there is a demand for more accurate warning technology to avoid accidents involving moving vehicles.

[0005] In view of the above-mentioned problems, an object of the present disclosure is to provide a warning device, system, method, and program for improving the accuracy of warnings regarding vehicles in motion.

[0006] A warning device according to a first aspect of the present disclosure includes an area recognition means for recognizing an area included in an image captured of the surroundings of a vehicle in motion; a first determination means for determining whether character recognition is necessary based on the type of the recognized area; a character recognition means for performing character recognition on the recognized area when it is determined that the character recognition is necessary; a second determination means for determining whether a warning is necessary based on the result of the character recognition and specification information of the vehicle; and an output means for outputting warning information when it is determined that the warning is necessary.

[0007] A warning system according to a second aspect of the present disclosure includes an imaging means for capturing images of the surroundings of a vehicle; an area recognition means for recognizing an area included in an image captured by the imaging means while the vehicle is traveling; a first determination means for determining whether character recognition is necessary based on the type of the recognized area; a character recognition means for performing character recognition on the recognized area when it is determined that the character recognition is necessary; a second determination means for determining whether a warning is necessary based on the result of the character recognition and specification information of the vehicle; and an output means for outputting warning information when it is determined that the warning is necessary.

[0008] A warning method according to a third aspect of the present disclosure includes a computer recognizing an area included in an image captured of the surroundings of a moving vehicle, determining whether character recognition is necessary based on the type of the recognized area, performing character recognition on the recognized area if it is determined that character recognition is necessary, determining whether a warning is necessary based on the result of the character recognition and specification information of the vehicle, and outputting warning information if it is determined that the warning is necessary.

[0009] A non-transitory computer-readable medium storing a warning program according to the fourth aspect of the present disclosure causes a computer to execute the following: an area recognition process that recognizes an area included in an image captured of the surroundings of a moving vehicle; a first determination process that determines whether character recognition is necessary based on the type of the recognized area; a character recognition process that performs character recognition on the recognized area if it is determined that character recognition is necessary; a second determination process that determines whether a warning is necessary based on the result of the character recognition and specification information of the vehicle; and an output process that outputs warning information if it is determined that the warning is necessary.

[0010] The present disclosure can provide a warning device, system, method, and program for improving the accuracy of warnings regarding a moving vehicle.

[0011] FIG. 1 is a block diagram showing the configuration of a warning device according to the first embodiment. FIG. 2 is a flowchart showing the flow of a warning method according to the first embodiment. FIG. 3 is a block diagram showing the configuration of a warning system according to the second embodiment. FIG. 4 is a flowchart showing the flow of warning processing according to the second embodiment. FIG. 5 is a diagram showing an example of a captured image according to the second embodiment. FIG. 6 is a diagram showing an example of a captured image according to the second embodiment. FIG. 7 is a flowchart showing the flow of warning processing according to the third embodiment. FIG. 8 is a flowchart showing the flow of warning processing according to the fourth embodiment. FIG. 9 is a block diagram showing the configuration of a warning system according to the fifth embodiment. FIG. 10 is a sequence diagram showing the flow of warning processing according to the fifth embodiment.

[0012] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In each drawing, the same or corresponding elements are designated by the same reference numerals, and for clarity of explanation, duplicate explanations will be omitted as necessary.

[0013] First Embodiment FIG. 1 is a block diagram showing the configuration of a warning device 1 according to the first embodiment. The warning device 1 is an information processing device for issuing a warning to avoid a collision between a vehicle and a road structure. The warning device 1 includes an area recognition unit 11, a first determination unit 12, a character recognition unit 13, a second determination unit 14, and an output unit 15. The area recognition unit 11 recognizes an area included in an image captured of the surroundings of a traveling vehicle. The area recognition process for recognizing an area divides a set of coordinates in image data into multiple areas based on the shape, pixel value, etc., and identifies the type of each area. The area recognition process may use publicly known technology. The first determination unit 12 determines whether character recognition is necessary based on the type of area recognized by the area recognition unit 11. If character recognition is determined to be necessary, the character recognition unit 13 performs character recognition on the recognized area. The character recognition process may use publicly known technology. The second determination unit 14 determines whether a warning is necessary based on the result of the character recognition and vehicle specification information. Here, the vehicle specification information is information indicating the vehicle specifications such as vehicle height, vehicle width, weight, etc. The output unit 15 outputs warning information when it is determined that a warning is necessary.

[0014] FIG. 2 is a flowchart showing the flow of the warning method according to the first embodiment. First, the area recognition unit 11 recognizes an area included in an image captured around a moving vehicle (S11). Next, the first determination unit 12 determines whether character recognition is necessary based on the type of the recognized area (S12). For example, the first determination unit 12 may determine that character recognition is necessary if the type of the recognized area corresponds to a specific type. If it is determined that character recognition is necessary in step S12, the character recognition unit 13 performs character recognition on the recognized area (S13). Then, the second determination unit 14 determines whether a warning is necessary based on the result of the character recognition and the vehicle specification information (S14). For example, if the result of the character recognition includes a threshold value for the specification information, the second determination unit 14 may determine that a warning is necessary if the corresponding vehicle specification information exceeds the threshold value. If it is determined that a warning is necessary in step S14, the output unit 15 outputs warning information (S15). If it is determined in step S12 or S14 that the warning information is not required, the warning information does not have to be output.

[0015] In this manner, in this embodiment, the necessity of character recognition is determined based on the type of area recognized in an image of the exterior of the vehicle captured while the vehicle is moving, and character recognition is performed after selecting images worthy of character recognition. This allows for efficient and meaningful character recognition, making it easier to accurately recognize constraint information in the vehicle specification information. This allows for more accurate determination of whether a warning is necessary, thereby improving the accuracy of warnings regarding the vehicle while it is moving.

[0016] The warning device 1 includes a processor, a memory, and a storage device (not shown). The storage device stores a computer program that implements the processing of the warning method according to this embodiment. The processor then loads the computer program from the storage device into the memory and executes the computer program. This allows the processor to implement the functions of an area recognition unit 11, a first determination unit 12, a character recognition unit 13, a second determination unit 14, and an output unit 15.

[0017] Alternatively, each component of the warning device 1 may be realized by dedicated hardware. Furthermore, some or all of the components of each device may be realized by general-purpose or dedicated circuits, processors, etc., or a combination of these. These may be configured by a single chip, or by multiple chips connected via a bus. Some or all of the components of each device may be realized by a combination of the above-mentioned circuits, etc., and a program. Furthermore, a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), an FPGA (Field-Programmable Gate Array), a quantum processor (quantum computer control chip), etc., may be used as the processor.

[0018] Furthermore, when some or all of the components of the warning device 1 are realized by multiple information processing devices, circuits, etc., the multiple information processing devices, circuits, etc. may be centrally or decentralized. For example, the information processing devices, circuits, etc. may be realized as a client-server system, a cloud computing system, or the like, connected via a communication network. Furthermore, the functions of the warning device 1 may be provided in the form of SaaS (Software as a Service).

[0019] Second Embodiment The second embodiment is a specific example of the first embodiment described above. FIG. 3 is a block diagram showing the configuration of a warning system according to the second embodiment. Here, a vehicle 1000 is shown as an example of a warning system. The vehicle 1000 includes a camera 100, a warning device 200, and an output device 300. The camera 100 is an example of an imaging means for capturing images of the surroundings of the vehicle 1000. The surroundings of the vehicle 1000 refer to the outside of the vehicle 1000, such as the front, rear, and sides of the vehicle 1000, and may include at least some of these. The camera 100 may be a so-called in-vehicle camera. The output device 300 is a device that outputs warning information generated by the warning device 200. The output device 300 includes, for example, a display device 310 and a speaker 320. The display device 310 is a display installed in the vehicle, and may be a display for a car navigation system. The speaker 320 outputs a warning sound. The output device 300 may also include a warning lamp, etc.

[0020] The warning device 200 is a so-called in-vehicle device, and is an example of the above-mentioned warning device 1. The warning device 200 is an information processing device that performs area recognition, character recognition, and depth recognition on an image of the outside of the vehicle captured by the camera 100, and outputs warning information via the output device 300 when a warning is necessary. The warning device 200 may be realized by an ECU (Electronic Control Unit) mounted on the vehicle 1000. Alternatively, the warning device 200 may be configured redundantly with multiple servers, and each functional block may be realized by multiple computers.

[0021] The warning device 200 includes a storage unit 210, a memory 220, an IF (Interface) unit 230, and a control unit 240. The storage unit 210 is an example of a storage device such as a hard disk or a flash memory. The storage unit 210 stores a program 211 and vehicle specification information 212. The program 211 is a computer program (warning program) that implements the warning process and the like according to the second embodiment. The vehicle specification information 212 is information that indicates the vehicle specifications of the vehicle 1000, such as the vehicle height, vehicle width, and weight.

[0022] The memory 220 is a volatile storage device such as a RAM (Random Access Memory), and is a storage area for temporarily storing information during operation of the control unit 240. The IF unit 230 is a communication interface between the inside of the warning device 200 and the camera 100 and output device 300.

[0023] The control unit 240 is a processor, i.e., a control device, that controls each component of the warning device 200. The control unit 240 loads the program 211 from the storage unit 210 into the memory 220 and executes the program 211. In this way, the control unit 240 realizes the functions of an acquisition unit 241, an area recognition unit 242, a determination unit 243, a character recognition unit 244, a depth recognition unit 245, and an output unit 246.

[0024] The acquisition unit 241 acquires images captured by the camera 100 while the vehicle 1000 is traveling. The acquisition unit 241 may acquire the traveling speed of the vehicle 1000 from a speedometer or the like of the vehicle 1000.

[0025] The area recognition unit 242 is an example of the area recognition unit 11 described above. The area recognition unit 242 performs area recognition on the image acquired by the acquisition unit 241. Here, the area recognition unit 242 divides the image into multiple areas by shape recognition, color identification, etc., based on the shape of a predetermined area, which is a collection of adjacent pixels in the image data, and the pixel values ​​of each pixel, and determines the type of each area. An area is a collection of two-dimensional coordinates of adjacent pixels in the image data. Examples of types include, but are not limited to, roads, sidewalks, pedestrians, streetlights, road signs, plates, viaducts, pedestrian bridges, and roadside trees.

[0026] The determination unit 243 is an example of the first determination unit 12 and the second determination unit 14 described above. The determination unit 243 determines whether the type of each area recognized by the area recognition unit 242 corresponds to a specific type. Here, the specific type refers to the type of area that may contain traffic restriction information, such as characters, symbols, or figures that restrict the passage of the vehicle 1000. Examples of the specific type include, but are not limited to, street lights, road signs, cards, viaducts, and pedestrian bridges. The specific types are a subset of the types that may be recognized by the area recognition unit 242 and may be set in advance in the storage unit 210. The determination unit 243 then determines that character recognition is necessary when the type of the recognized area corresponds to a specific type. This is because, when the type of the area is a street light, road sign, card, viaduct, pedestrian bridge, or the like, there is a high probability that traffic restriction information is written therein, thereby increasing the accuracy of character recognition.

[0027] Furthermore, the determination unit 243 determines whether or not a warning is necessary based on at least one of a character recognition result by the character recognition unit 244 (described later) and a depth recognition result by the depth recognition unit 245 (described later), and the vehicle specification information 212. If a character recognition result is present, the determination unit 243 determines that a warning is necessary if the vehicle specification information 212 exceeds a threshold included in the character recognition result. If a depth recognition result is present, the determination unit 243 determines that a warning is necessary if the vehicle specification information 212 exceeds a threshold of the specification information calculated from the depth recognition result. Furthermore, the determination unit 243 may determine whether or not a warning is necessary based on the character recognition result, the depth recognition result, the area recognition result, and the vehicle specification information 212. Note that the determination unit 243 may determine whether or not a warning is necessary by taking into account the traveling speed of the vehicle 1000 in addition to the above.

[0028] The character recognition unit 244 is an example of the above-mentioned character recognition unit 13. The character recognition unit 244 performs character recognition on an area corresponding to a specific type, and outputs the character recognition result to the determination unit 243.

[0029] The depth recognition unit 245 performs depth recognition on the area recognized by the area recognition unit 242. Here, depth recognition refers to a process of calculating the distance between each portion of the image data and the recognition object. The depth recognition unit 245 may, for example, calculate the distance between the camera 100 and the recognition object as a depth based on the color shading of each pixel in the image data. The depth recognition process may use publicly known technology. Furthermore, the depth recognition unit 245 may calculate a threshold value for the specification information from the calculated depth and the area recognition result and output the threshold value to the determination unit 243 as the depth recognition result. In other words, the depth recognition unit 245 may combine the depth recognition result and the area recognition result to recognize the three-dimensional positional relationship and distance between the vehicle 1000 and the recognition object area (e.g., the recognition object) by combining the depth recognition result and the area recognition result. The depth recognition unit 245 may, for example, calculate the height, width, etc. of the recognition object from the recognized distance and use these to calculate the threshold value for the specification information.

[0030] Furthermore, the depth recognition unit 245 may perform depth recognition on recognized areas within a predetermined range including the center of the image. This allows priority to be given to depth recognition on areas where the vehicle 1000 is more likely to collide, thereby avoiding recognition processing on areas where the collision is less likely, thereby improving the efficiency of the recognition processing. Furthermore, the depth recognition unit 245 performs depth recognition regardless of whether character recognition is required. Alternatively, the depth recognition unit 245 may perform depth recognition when the type of the recognized area corresponds to a specific type. In other words, the depth recognition unit 245 may perform depth recognition on the target area for character recognition.

[0031] The output unit 246 is an example of the above-mentioned output unit 15. The output unit 246 generates warning information based on the region recognition result, the character recognition result, and the depth recognition result, and outputs the warning information via the output device 300. For example, when character recognition is performed, the output unit 246 may generate and output warning information according to the character recognition result.

[0032] 4 is a flowchart showing the flow of warning processing according to the second embodiment. As a premise, the camera 100 captures an image of the exterior of the vehicle 1000 while the vehicle 1000 is traveling and outputs the captured image data to the warning device 200. In response, the acquisition unit 241 of the warning device 200 acquires an image of the surroundings of the traveling vehicle 1000 (S201). The area recognition unit 242 then performs area recognition on the acquired image (S202). At this time, the area recognition unit 242 divides the image data into multiple areas (identifies the areas) based on the shape, color, etc. of the display content within the image, and recognizes the type of each area. In other words, the area recognition unit 242 assigns a recognized type (e.g., a label) to each of the multiple recognized areas (a set of two-dimensional coordinates).

[0033] Then, the determination unit 243 determines whether the type of each recognized area is a specific type (S203). That is, the determination unit 243 determines whether the type assigned to each of the multiple areas is a predetermined specific type. In step S203, for areas among the multiple areas determined to be of a specific type, step S204 is processed, and for areas determined to be of a type other than the specific type, the process proceeds to step S205.

[0034] Next, the character recognition unit 244 performs character recognition on the recognized area (the area determined to be of a specific type) (S204).The character recognition unit 244 then outputs the character string recognized from the area to the determination unit 243 as the character recognition result.

[0035] FIG. 5 is a diagram showing an example of a captured image 41 according to the second embodiment. The image 41 is an example of an image captured in the traveling direction of the vehicle 1000. Within the image 41, an overpass 412 is shown as an example of a recognized area. The overpass 412 is an example of an area determined to be of the specific type "overpass." The area of ​​the overpass 412 includes a plate 411. The plate 411 is an example of an area determined to be of the specific type "road sign." The plate 411 shows that character strings corresponding to traffic restriction information, such as "Demobilization 3.0M" and "Beam height 1.8M," are written on it.

[0036] In the example of FIG. 5 , the character recognition unit 244 performs character recognition processing on the region of the viaduct 412 or the plate 411. As a result, the character recognition unit 244 recognizes character strings such as "Demobilization 3.0M" and "Beam Depth 1.8M," and from the recognized character strings, recognizes the vehicle width threshold value of "3.0M" and the vehicle height threshold value of "1.8M" in the specification information. The character recognition unit 244 then outputs, for example, the vehicle width threshold value of "3.0M" and the vehicle height threshold value of "1.8M" to the determination unit 243 as character recognition results. Note that the character recognition unit 244 does not perform character recognition processing on regions other than the viaduct 412 in the image 41, thereby suppressing unnecessary recognition processing and improving the accuracy of character recognition. Furthermore, by performing character recognition processing on the region of the plate 411, the character recognition unit 244 achieves higher character recognition accuracy compared to character recognition processing on the viaduct 412.

[0037] For the area where character recognition processing was performed in step S204 and the area determined to be other than a specific type in step S203, the determination unit 243 determines whether the recognized area is within a predetermined range including the center of the image (S205).

[0038] The depth recognition unit 245 performs depth recognition on the corresponding region (S206). FIG. 6 is a diagram showing an example of a captured image 42 according to the second embodiment. The image 42 is an example of an image captured in the traveling direction of the vehicle 1000. Within the image 42, a sign 421 and an overpass 423 are illustrated as examples of recognized regions. A central region 422 is an example of a predetermined range including the center of the image 42. Therefore, the depth recognition unit 245 performs depth recognition on each of the sign 421 and the overpass 423 and outputs the depth recognition results to the determination unit 243. At this time, the depth recognition unit 245 calculates the distance between the sign 421 or the overpass 423 and the vehicle 1000, and calculates the height of the sign 421 or the overpass 423 based on the position of the region in the image 42 on the two-dimensional coordinate system and the calculated distance. The depth recognition unit 245 uses the calculated height as a threshold for the specification information. In the example of Figure 6, the depth recognition unit 245 calculates the height of sign 421 from the road to be "3.0 M", calculates the height of viaduct 423 from the road to be "3.0 M", sets the vehicle height threshold to "3.0 M", and outputs this to the determination unit 243 as the depth recognition result.

[0039] For the area where the depth recognition process was performed in step S206 and the area determined to be outside the predetermined range in step S205, the determination unit 243 determines whether a warning is necessary based on at least one of the character recognition result and the depth recognition result (S207). Specifically, the determination unit 243 determines whether a warning is necessary based on the character recognition result, the depth recognition result, the area recognition result, and the vehicle specification information 212. Furthermore, the determination unit 243 may determine whether a warning is necessary by taking into account the traveling speed of the vehicle 1000.

[0040] If it is determined in step S207 that a warning is necessary, the output unit 246 generates warning information based on at least one of the character recognition result and the depth recognition result (S208). Specifically, the output unit 246 may generate a message indicating a warning or a request for a detour, including a threshold value of the specification information included in the character recognition result, as the warning information. Note that the warning information may be a warning sound, or may be both a warning sound and a message. The output unit 246 outputs the warning information to the output device 300 (S209).

[0041] 5 , the output unit 246 may generate a message such as "Vehicle width limit: 3.0 m, vehicle height limit: 1.8 m, please drive carefully" as warning information, and the display device 310 may display the message on the screen. Alternatively, the output unit 246 may generate a message such as "Vehicle width limit: 3.0 m, vehicle height limit: 1.8 m, this vehicle cannot pass through this area, please take a detour" and a warning sound as warning information, and the display device 310 may display the message on the screen, and the speaker 320 may output the warning. Note that after step S209 or if it is determined in step S207 that a warning is not necessary, the warning process may be terminated, or the process may return to step S201 and continue the warning process.

[0042] As described above, this embodiment can improve the accuracy of warnings regarding moving vehicles, similar to the first embodiment. In particular, this embodiment can improve the accuracy of warnings because it uses depth recognition results in addition to character recognition results to determine whether a warning is necessary. Furthermore, even if it is determined based on the area recognition results that character recognition is unnecessary (NO in step S203), depth recognition is performed, so it is possible to accurately determine whether a warning is necessary using the area recognition results and the depth recognition results. Furthermore, because depth recognition is performed within a predetermined range within the image, it is possible to reduce the load of depth recognition processing in areas where the probability of a vehicle collision is low.

[0043] <Embodiment 3> This embodiment 3 is a modified example of the above-described embodiment 2. The depth recognition unit according to this embodiment 3 performs depth recognition when it is determined that character recognition is unnecessary. Note that other configurations and processes according to this embodiment 3 are the same as those of the above-described embodiment 2, and therefore, redundant explanations and illustrations will be omitted.

[0044] 7 is a flowchart showing the flow of the warning process according to the third embodiment. In FIG. 7, unlike FIG. 4 described above, if the type of the area recognized in step S203 is not a specific type, the depth recognition unit 245 performs depth recognition on the corresponding area (the area that is not a specific type) (S206a). The specific process of step S206a is the same as that of step S206 described above.

[0045] After step S204 or S206a, the determination unit 243 determines whether a warning is necessary based on the character recognition result or the depth recognition result (S207a). If it is determined that a warning is necessary, the output unit 246 performs the above-mentioned steps S208 and S209. If it is determined that a warning is not necessary, the output unit 246 ends the warning process or returns to step S201, as after step S209.

[0046] In this way, in the third embodiment, depth recognition is performed even when it is determined based on the area recognition result that character recognition is unnecessary, so that it is possible to accurately determine whether or not a warning is necessary using the area recognition result and the depth recognition result. In addition, the third embodiment can achieve the same effects as the second embodiment described above.

[0047] <Fourth Embodiment> This fourth embodiment is a modification of the second or third embodiment described above. When it is determined that character recognition is necessary, the character recognition unit according to this fourth embodiment performs character recognition using a recognition algorithm corresponding to the type of recognized area. Therefore, for example, the storage unit 210 of the warning device 200 stores a corresponding recognition algorithm (identification information) associated with each of multiple specific types. Furthermore, the character recognition unit 244 is capable of executing multiple types of specified recognition algorithms. Note that other configurations and processes according to this fourth embodiment are similar to those of the second embodiment described above, and therefore redundant explanations and illustrations will be omitted.

[0048] FIG. 8 is a flowchart showing the flow of the warning process according to the fourth embodiment. Compared to FIG. 4 described above, FIG. 8 adds step S204b before step S204. Specifically, if an area is determined to be one of multiple specific types in step S203, the character recognition unit 244 selects a recognition algorithm according to the recognized (determined) type (S204b). Then, the character recognition unit 244 performs character recognition on the area recognized as the specific type using the selected recognition algorithm (S205). The subsequent processing is the same as that described in FIG. 4 above. Furthermore, step S204b may be executed before step S204 in FIG. 7 described above.

[0049] In this way, in the fourth embodiment, a recognition algorithm with high recognition efficiency is set in advance for the type of area recognized, and character recognition can be performed using an appropriate recognition algorithm corresponding to the specific recognized type. For example, character recognition processing specialized in recognizing the position of the plate 411 in the viaduct 412 and the character string written on it, or recognition processing specialized in recognizing the outline (circle), symbol, shape, and color of the sign 421 and the character string written on it, can be used separately. This reduces the recognition processing load and further improves recognition efficiency and recognition accuracy.

[0050] <Embodiment 5> The present embodiment 5 is another specific example of the above-described embodiment 1, and is a modification of embodiments 2 to 4. The warning device according to the present embodiment 5 is installed outside a vehicle and communicates wirelessly with the vehicle.

[0051] 9 is a block diagram showing the configuration of a warning system according to the fifth embodiment. In the warning system, a vehicle 1000c and a warning device 200c are connected via a network N. The network N is a communication network including a wireless communication line. Specifically, the network N includes, for example, the Internet, a wireless communication line network, a mobile phone line network, etc. Furthermore, the network N does not matter what type of communication protocol is used.

[0052] The vehicle 1000c includes a camera 100, a wireless communication unit 400, and an output device 300. The camera 100 and the output device 300 are the same as those shown in Fig. 3. The wireless communication unit 400 wirelessly transmits images captured by the camera 100 to the warning device 200c via the network N, and outputs warning information received from the warning device 200c via the network N via wireless communication to the output device 300.

[0053] The warning device 200c is different from the above-described warning device 200 in that a program 211c, an acquisition unit 241c, and an output unit 246c are replaced. The other components are the same as those of the above-described warning device 200, and therefore descriptions thereof will be omitted as appropriate. The program 211c is a computer program (warning program) in which the warning processing and the like according to the fifth embodiment are implemented.

[0054] The acquisition unit 241c acquires images captured by the camera 100 while the vehicle 1000c is traveling from the vehicle 1000c by wireless communication via the network N. The acquisition unit 241c may also acquire the traveling speed of the vehicle 1000c.

[0055] The output unit 246c outputs the warning information to the vehicle 1000c via the network N by wireless communication in order to cause the output device 300 to output the warning information.

[0056] FIG. 10 is a sequence diagram showing the flow of the warning process according to the fifth embodiment. First, the camera 100 of the vehicle 1000c captures an image of the surroundings of the vehicle 1000c while the vehicle 1000c is traveling (S301). Then, the wireless communication unit 400 of the vehicle 1000c transmits the captured image to the warning device 200c via the network N (S302). In response, the acquisition unit 241c of the warning device 200c acquires the captured image from the warning device 200c. Then, the warning device 200c executes the warning process (S303), similar to steps S202 to S209 of FIG. 4 described above. Then, the output unit 246c of the warning device 200c transmits the generated warning information to the vehicle 1000c via the network N (S304). In response, the wireless communication unit 400 of the vehicle 1000c receives the warning information from the warning device 200c via the network N. Then, the output device 300 outputs the received warning information (S305).

[0057] As described above, this embodiment can improve the accuracy of warnings regarding a moving vehicle, similar to the first embodiment described above. In particular, in this embodiment, a server installed outside the vehicle can be used as the warning device, rather than an in-vehicle device. This reduces the processing load on the in-vehicle device. Furthermore, when the vehicle is driven autonomously, it becomes easier to add or change advanced processing on the server side. This makes it easier to introduce the warning device according to this embodiment.

[0058] Other Embodiments The above-described embodiments may also have the following configuration. For example, the output unit may output the image capture location and warning information when it is determined that a warning is necessary, in order to generate map information for warning that is to be provided to other vehicles whose specification information is similar to that of the vehicle within a predetermined range. For example, the acquisition unit may acquire the capture location of the acquired image along with the image. The output unit may then register a pair of location information indicating the capture location and the warning information in a server for generating the map information for warning or a database for generating the map information for warning. The output unit may then register the pair of location information and warning information in real time while the vehicle is traveling or when the vehicle is parked or stopped. This allows map information that accumulates information on locations where the probability of vehicle collision is high to be created by combining data such as the heights of viaducts, road signs, roadside trees, etc., and the width of roads (tunnels) collected during actual vehicle travel with the location information. This map information may then be shared between vehicles with similar specification information, enabling an early warning to be issued when approaching a location with a high probability of collision before image recognition processing is performed, thereby supporting earlier and more precise collision avoidance.

[0059] Furthermore, if it is determined that a warning is necessary, the output unit may output warning information to the outside of the vehicle. This allows the warning information to be shared with pedestrians, cyclists, vehicles, etc. around the vehicle, or the warning to be notified. For example, if the vehicle realizes that it cannot pass under an overpass ahead and attempts to make a sudden stop or a sharp turn, it becomes easier to prevent collisions with surrounding vehicles.

[0060] Other Embodiments In the above examples, the program includes instructions (or software code) that, when loaded into a computer, cause the computer to perform one or more functions described in the embodiments. The program may be stored on a non-transitory computer-readable medium or a tangible storage medium. By way of example and not limitation, computer-readable media or tangible storage media include random-access memory (RAM), read-only memory (ROM), flash memory, solid-state drive (SSD) or other memory technology, CD-ROM, digital versatile disc (DVD), Blu-ray® disc or other optical disk storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage device. The program may also be transmitted on a transitory computer-readable medium or communication medium. By way of example and not limitation, transitory computer-readable media or communication media include electrical, optical, acoustic, or other forms of propagated signals.

[0061] The present disclosure is not limited to the above-described embodiments, and may be modified as appropriate without departing from the spirit and scope of the present disclosure. In addition, the present disclosure may be implemented by appropriately combining the respective embodiments.

[0062] Some or all of the above embodiments may be described as, but are not limited to, the following supplementary notes. (Supplementary Note A1) A warning device comprising: area recognition means for recognizing an area included in an image captured of the surroundings of a moving vehicle; first determination means for determining whether character recognition is necessary based on the type of the recognized area; character recognition means for performing character recognition on the recognized area if it is determined that character recognition is necessary; second determination means for determining whether a warning is necessary based on the result of the character recognition and specification information of the vehicle; and output means for outputting warning information if it is determined that the warning is necessary. (Supplementary Note A2) The warning device according to Supplementary Note A1, further comprising depth recognition means for performing depth recognition on the recognized area, wherein the second determination means further determines whether the warning is necessary based on the result of the depth recognition. (Supplementary Note A3) The warning device according to Supplementary Note A2, wherein the depth recognition means performs the depth recognition if it is determined that the character recognition is unnecessary. (Appendix A4) The warning device according to Appendix A2 or A3, wherein the depth recognition means performs the depth recognition on an area within a predetermined range including the center of the image out of the recognized area. (Appendix A5) The warning device according to any one of Appendix A2 to Appendix A4, wherein the depth recognition means performs the depth recognition when the type of the recognized area corresponds to a specific type. (Appendix A6) The warning device according to any one of Appendix A1 to Appendix A5, wherein the first determination means determines that the character recognition is necessary when the type of the recognized area corresponds to a specific type. (Appendix A7) The warning device according to any one of Appendix A1 to Appendix A6, wherein the output means outputs the shooting position of the image and the warning information when it is determined that the warning is necessary, in order to generate map information for a warning to be provided to other vehicles whose specification information is similar to that of the vehicle within a predetermined range. (Appendix A8) The warning device according to any one of appendices A1 to A7, wherein the output means outputs the warning information to an outside of the vehicle when it is determined that the warning is necessary.(Appendix A9) The warning device according to any one of Appendices A1 to A8, wherein the character recognition means, when it is determined that the character recognition is necessary, performs the character recognition using a recognition algorithm according to the type of the recognized area. (Appendix A10) The warning device according to any one of Appendices A1 to A9, wherein the output means, when it is determined that the warning is necessary, outputs the warning information according to the result of the character recognition. (Appendix B1) A warning system comprising: an imaging means for imaging an area around a vehicle; an area recognition means for recognizing an area included in an image captured by the imaging means while the vehicle is traveling; a first determination means for determining whether character recognition is necessary based on the type of the recognized area, when it is determined that the character recognition is necessary, a character recognition means for performing character recognition on the recognized area, a second determination means for determining whether a warning is necessary based on the result of the character recognition and specification information of the vehicle, and an output means for outputting warning information when it is determined that the warning is necessary. (Appendix B2) The warning system according to Appendix B1, further comprising: depth recognition means for performing depth recognition on the recognized area, wherein the second determination means determines whether or not the warning is necessary taking into account the result of the depth recognition. (Appendix C1) A warning method, wherein a computer recognizes an area included in an image of the surroundings of a traveling vehicle, determines whether or not character recognition is necessary based on the type of the recognized area, performs character recognition on the recognized area if it is determined that the character recognition is necessary, determines whether or not a warning is necessary based on the result of the character recognition and specification information of the vehicle, and outputs warning information if it is determined that the warning is necessary.(Appendix D1) A non-transitory computer-readable medium storing a warning program that causes a computer to execute the following: an area recognition process that recognizes an area included in an image captured of the surroundings of a moving vehicle; a first determination process that determines whether character recognition is necessary based on the type of the recognized area; a character recognition process that performs character recognition on the recognized area if it is determined that character recognition is necessary; a second determination process that determines whether a warning is necessary based on the result of the character recognition and specification information of the vehicle; and an output process that outputs warning information if it is determined that the warning is necessary.

[0063] Although the present invention has been described above with reference to the embodiments (and examples), the present invention is not limited to the above-described embodiments (and examples). Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.

[0064] 1 Warning device 11 Area recognition unit 12 First determination unit 13 Character recognition unit 14 Second determination unit 15 Output unit 1000 Vehicle 1000c Vehicle 100 Camera 200 Warning device 200c Warning device 210 Storage unit 211 Program 211c Program 212 Vehicle specification information 220 Memory 230 IF unit 240 Control unit 241 Acquisition unit 241c Acquisition unit 242 Area recognition unit 243 Determination unit 244 Character recognition unit 245 Depth recognition unit 246 Output unit 246c Output unit 300 Output device 310 Display device 320 Speaker 400 Wireless communication unit N Network 41 Image 411 Plate 412 Viaduct 42 Image 421 Sign 422 Central area 423 viaduct

Claims

1. Region recognition means for recognizing a region included in an image obtained by photographing the surroundings of a vehicle during travel; First determination means for determining whether character recognition is necessary based on the type of the recognized region; Character recognition means for performing character recognition on the recognized region when it is determined that the character recognition is necessary; Second determination means for determining whether a warning is necessary based on the result of the character recognition and the vehicle specifications information; Output means for outputting warning information when it is determined that the warning is necessary; A warning device comprising the above.

2. Further comprising depth recognition means for performing depth recognition on the recognized region, wherein the second determination means further determines whether the warning is necessary based on the result of the depth recognition. The warning device according to claim 1.

3. The depth recognition means, performs the depth recognition when it is determined that the character recognition is unnecessary. The warning device according to claim 2.

4. The depth recognition means, performs the depth recognition on a region within a predetermined range including the center of the image among the recognized regions. The warning device according to claim 2.

5. The depth recognition means, performs the depth recognition when the type of the recognized region corresponds to a specific type. The warning device according to claim 2.

6. The first determination means, determines that the character recognition is necessary when the type of the recognized region corresponds to a specific type. The warning device according to any one of claims 1 to 5.

7. The output means, outputs the imaging position of the image and the warning information when it is determined that the warning is necessary in order to generate warning map information for providing to another vehicle whose specifications information is similar to that of the vehicle within a predetermined range. The warning device according to any one of claims 1 to 5.

8. Imaging means for photographing the surroundings of a vehicle; Region recognition means for recognizing a region included in an image photographed by the imaging means during travel of the vehicle; First determination means for determining whether character recognition is necessary based on the type of the recognized region; Character recognition means for performing character recognition on the recognized region when it is determined that the character recognition is necessary; Second determination means for determining whether a warning is necessary based on the result of the character recognition and the vehicle specifications information; Output means for outputting warning information when it is determined that the warning is necessary; A warning system comprising the above.

9. A computer, recognizes a region included in an image obtained by photographing the surroundings of a vehicle during travel, Based on the type of the recognized area, determine whether character recognition is necessary, if it is determined that the character recognition is necessary, perform character recognition on the recognized area, based on the result of the character recognition and the vehicle specifications information, determine whether a warning is necessary, if it is determined that the warning is necessary, output warning information Warning method.

10. an area recognition process for recognizing an area included in an image of the surroundings of a vehicle during travel, a first determination process for determining whether character recognition is necessary based on the type of the recognized area, a character recognition process for performing character recognition on the recognized area if it is determined that the character recognition is necessary, a second determination process for determining whether a warning is necessary based on the result of the character recognition and the vehicle specifications information, an output process for outputting warning information if it is determined that the warning is necessary, A warning program that causes a computer to execute.