Dead angle area extraction device, dead angle area extraction method, and dead angle area extraction program

Abstract

To execute appropriate control of a mobile while taking a change tendency of a dead angle area into account.SOLUTION: A processor of a dead angle area extraction device stores in a time-sequential order at least first map data including a first dead angle area to be a dead angle due to an object in a case where the object is watched from a mobile being present at a first specific position and second map data including a second dead angle area to be a dead angle due to an object in a case where the object is watched from a mobile being present at a second position different from the first specific position, and extracts a pattern which is an image indicating time-sequential changes of the first dead angle area and the second dead angle area on the basis of the first map data and the second map data.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to a blind spot area extraction device, a blind spot area extraction method, and a blind spot area extraction program. [Background technology]

[0002] Patent Document 1 discloses a technology for estimating the risk that an obstacle in a blind spot area will jump out in front of a moving body in a specific position when the moving body sees an object that creates a blind spot. Specifically, the conventional technology of Patent Document 1 calculates multiple blind spot areas detected at each position of a moving body, which is a vehicle, when the moving body travels, and estimates a potential risk by setting obstacles that may be present in the blind spot areas and the speed at which the obstacles will enter the vehicle based on changes in the area and shape of each blind spot area. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-144036 A [Non-patent literature]

[0004] [Non-Patent Document 1] "Lidar to grid", searched June 29, 2023, Internet <URL:https: / / atsushisakai.github.io / PythonRobotics / modules / mapping / lidar_to_grid_map_tutorial / lidar_to_grid_map_tutorial.html> [Non-Patent Document 2] "Updating Occupancy Probability Using Bayes' Theorem," retrieved June 29, 2023, Internet<URL:https: / / www.eureka-moments-blog.com / entry / 2022 / 07 / 25 / 170130> [Non-Patent Document 3] "3.2.1. Occupancy Grid Map," retrieved June 29, 2023, Internet<URL:https: / / www.coursera.org / lecture / robotics-learning / 3-2-1-occupancy-grid-map-0QuFW> [Non-Patent Document 4] "Multiresolution Gray Scale and Rotation Invariant Texture Classification with Local Binary Patterns", retrieved June 29, 2023, Internet <URL:https: / / www.google.com / search?q=%5B5%5D+T.+Ojala%2C+M.+Pietikainen%2C+and+T.+Maenpaa%2C+%E2%80%9CMultiresolution+gray-scale+and+rotation+invariant+texture+classification+with+local+binary+patterns%2C%E2%80%9D+IEEE+Transactions+on+pattern+analysis+and+machine+intelligence%2C+vol.+24%2C&rlz=1C1YTUH_jaJP1041JP1043&oq=%5B5%5D+T.+Ojala%2C+M.+Pietikainen%2C+and+T.+Maenpaa%2C+%E2%80%9CMultiresolution+gray-scale+and+rotation+invariant+texture+classification+with+local+binary+patterns%2C%E2%80%9D+IEEE+Transactions+on+pattern+analysis+and+machine+intelligence%2C+vol.+24%2C&aqs=chrome..69i57.767j0j4&sourceid=chrome&ie=UTF-8> [Non-Patent Document 5] "The Recognition of Human Movement Using Temporal Templates," retrieved June 29, 2023, Internet <URL:https: / / www.semanticscholar.org / paper / The-Recognition-of-Human-Movement-Using-Temporal-Bobick-Davis / 886431a362bfdbcc6dd518f844eb374950b9de86> Summary of the Invention [Problem to be solved by the invention]

[0005] Here, the depth of the blind spot area, i.e., the depth and width of the blind spot area, may change depending on the moving path of the moving object. In the conventional technology, the changing tendency of the depth of the blind spot area is not taken into consideration, so there is a possibility that it is not possible to appropriately control the operation of the moving object corresponding to the predicted movement of the obstacle, or to appropriately control the timing of issuing an alarm to the obstacle. Thus, the conventional technology has room for improvement in controlling the moving object while taking into consideration the changing tendency of the blind spot area.

[0006] The present disclosure has been made in consideration of the above-mentioned facts, and aims to provide a blind spot area extraction device, a blind spot area extraction method, and a blind spot area extraction program that can perform appropriate control of a moving body taking into account the changing trends of blind spot areas. [Means for solving the problem]

[0007] In order to achieve the above-mentioned object, a blind spot area extraction device according to a first aspect is a blind spot area extraction device having at least one processor, wherein the processor acquires multiple pieces of distance information indicating the distance from a distance sensor mounted on a moving body to objects around the moving body in chronological order as the moving body moves, and accumulates, based on the multiple pieces of distance information, in chronological order, first map data including at least a first blind spot area that is a blind spot caused by an object when the object is viewed from the moving body located at a first specific position, and second map data including a second blind spot area that is a blind spot caused by the object when the object is viewed from the moving body located at a second position different from the first specific position, and extracts a pattern that is an image showing changes over time in the first blind spot area and the second blind spot area based on the first map data and the second map data.

[0008] According to the first aspect, by using the extracted pattern, it is possible to predict the change tendency of the blind spot area when an object causing a blind spot is seen from a moving body moving along various moving paths. Therefore, when the depth of the blind spot that changes in chronological order is predicted to be shallower than a specific value, it is possible to determine that the degree to which an obstacle in the blind spot may come into contact with the moving body is low, and to suppress the reduction of the moving body's speed and to delay the timing of generating an alarm sound from the moving body. When the depth of the blind spot that changes in chronological order is predicted to be deeper than a specific value, it is possible to determine that the degree to which an obstacle in the blind spot may come into contact with the moving body is high, and to reduce the moving body's speed a certain distance before the object causing the blind spot and to advance the timing of generating an alarm sound from the moving body.

[0009] The processor of the blind spot extraction device according to the second aspect extracts the first blind spot area and the second blind spot area included in the pattern, and sets parameters corresponding to the depths of the first blind spot area and the second blind spot area based on the respective shapes of the extracted first blind spot area and the extracted second blind spot area.

[0010] According to the second aspect, by utilizing a parameter set corresponding to the depth of the blind spot area depending on the difference in the moving path of the moving body, it is possible to perform appropriate control of the moving body taking into account the tendency of the blind spot area to change due to the difference in the moving path of the moving body.

[0011] The pattern by the blind spot extraction device according to the third aspect is an image showing areas in which the first blind spot area and the second blind spot area have different color intensities.

[0012] According to the third aspect, the difference between the blind spots can be clearly distinguished.

[0013] The processor of the blind spot extraction device according to the fourth aspect extracts the pattern by utilizing features of a Local Binary Pattern (LBP) calculated based on pixel values ​​of a specific pixel included in the pattern, which is an image showing time-series changes in the first blind spot area and the second blind spot area, and surrounding pixels around the specific pixel.

[0014] According to the fourth aspect, by using the feature amount of the LBP, the effect of the change in illuminance around the moving object is reduced, and the pattern can be extracted with high accuracy.

[0015] The processor of the blind spot area extraction device according to the fifth aspect regards a pattern, which is an image showing time-series changes in the first blind spot area and the second blind spot area, as a collection of local features based on BoVW (Bag of Visual Words), converts the local features into a histogram, and extracts the pattern by searching the histogram using machine learning.

[0016] According to the fifth aspect, by searching a histogram of local feature amounts in a pattern, which is an image showing time-series changes in the first blind spot area and the second blind spot area, it is possible to extract the pattern with high accuracy.

[0017] The distance sensor of the blind spot area extraction device according to the sixth aspect includes a LiDAR (Light Detection And Ranging).

[0018] According to the sixth aspect, by using a general-purpose LiDAR, it is possible to maintain high distance resolution since distance measurement can be performed using infrared rays with short wavelengths, particularly laser light.

[0019] A blind spot area extraction method according to the seventh aspect is executed by a computer to acquire multiple pieces of distance information indicating the distance from a distance sensor mounted on a moving body to objects around the moving body in chronological order as the moving body moves, and based on the multiple pieces of distance information, accumulate in chronological order first map data including at least a first blind spot area caused by the object when the object is viewed from the moving body located at a first specific position, and second map data including a second blind spot area caused by the object when the object is viewed from the moving body located at a second position different from the first specific position, and extract a pattern, which is an image showing changes over time in the first blind spot area and the second blind spot area, based on the first map data and the second map data.

[0020] According to the seventh aspect, it becomes possible to predict, by means of a computer, the tendency of a blind spot area to change when an object causing a blind spot is seen from a moving body moving along various moving routes.

[0021] A blind spot area extraction program according to an eighth aspect causes a computer to acquire multiple pieces of distance information indicating the distance from a distance sensor mounted on a moving body to objects around the moving body in chronological order as the moving body moves, and based on the multiple pieces of distance information, accumulate in chronological order first map data including at least a first blind spot area that is a blind spot caused by an object when the object is viewed from the moving body located at a first specific position, and second map data including a second blind spot area that is a blind spot caused by the object when the object is viewed from the moving body located at a second position different from the first specific position, and extract a pattern that is an image showing changes over time in the first blind spot area and the second blind spot area based on the first map data and the second map data.

[0022] According to the eighth aspect, it becomes possible for a computer to predict the tendency of changes in blind spot areas when an object causing a blind spot is seen from a moving body moving along various moving routes. Effect of the Invention

[0023] As described above, according to the present invention, it is possible to provide a blind spot area extraction device, a blind spot area extraction method, and a blind spot area extraction program that are capable of executing appropriate control of a moving object taking into account the changing tendency of the blind spot area. [Brief description of the drawings]

[0024] [Figure 1] FIG. 1 is a diagram showing a configuration of a blind spot extraction system according to an embodiment of the present disclosure. [Diagram 2] FIG. 2 is a diagram showing a hardware configuration of a moving object. [Diagram 3] FIG. 3 is a diagram illustrating a hardware configuration of the center server. [Figure 4] FIG. 4 is a block diagram showing the functional configuration of the center server. [Diagram 5] FIG. 5 is a diagram for explaining a blind spot area when objects around a stationary moving body are viewed from the moving body. [Figure 6] FIG. 6 is a diagram for explaining a blind spot area when objects around a moving object are viewed from the moving object. [Figure 7] FIG. 7 is a diagram showing an example of an occupancy grid map. [Figure 8] FIG. 8 is a diagram for explaining how the change history of the occupancy grid map is accumulated in a time-series change map. [Figure 9] FIG. 9 is a diagram for explaining how the change history of the occupancy grid map is accumulated in a time-series change map. [Figure 10] FIG. 10 is a diagram for explaining the process of similarity evaluation. [Figure 11] FIG. 11 is a diagram for explaining an example of a window W having a characteristic pattern. [Figure 12] FIG. 12 shows the results of the similarity evaluation. [Figure 13] FIG. 13 is a diagram showing an example of a pattern. [Figure 14] FIG. 14 is a flowchart for explaining the operation of the blind spot area extraction system. [Figure 15] FIG. 15 is a diagram for explaining the function of determining the depth of the blind spot area. [Figure 16] FIG. 16 is a flowchart for explaining another operation of the blind spot area extraction system. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0025] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. In each drawing, the same or equivalent components and parts are denoted by the same reference numerals. In addition, the dimensional ratios of the drawings are exaggerated for the convenience of explanation and may differ from the actual ratios.

[0026] 1 is a diagram showing a configuration of a blind spot area extraction system according to an embodiment of the present disclosure. The blind spot area extraction system 10 includes a moving body 11, a center server 30, and an information providing server 40. The moving body 11 may be interpreted as a mobile robot, an automobile, or the like. The center server 30 may be interpreted as the blind spot area extraction device of the present disclosure.

[0027] The mobile object 11 and the center server 30 are connected to each other so as to be able to communicate with each other through a network N1. The center server 30 and the information providing server 40 are connected to each other so as to be able to communicate with each other through a network N2. The center server 30 and the information providing server 40 may be connected to each other through the network N1.

[0028] Next, the hardware configuration of the mobile object 11 will be described with reference to Fig. 2. Fig. 2 is a diagram showing the hardware configuration of the mobile object. The mobile object 11 includes an in-vehicle device 20, a plurality of ECUs (Electronic Control Units) 22, a plurality of in-vehicle devices 23, a distance sensor 25, and a GPS (Global Positioning System) device 29.

[0029] The vehicle-mounted device 20 includes a central processing unit (CPU) 20A, a read only memory (ROM) 20B, a random access memory (RAM) 20C, an in-vehicle communication interface (I / F) 20D, a wireless communication interface (I / F) 20E, and an input / output interface (I / F) 20F. The CPU 20A, the ROM 20B, the RAM 20C, the in-vehicle communication interface (I / F) 20D, the wireless communication interface (I / F) 20E, and the input / output interface (I / F) 20F are connected to each other so as to be able to communicate with each other via an internal bus 20G.

[0030] The CPU 20A may execute various programs and control various functions. The CPU 20A may read out programs from the ROM 20B and execute the programs using the RAM 20C as a working area. The ROM 20B may store various programs and various data. Specifically, the ROM 20B may store a control program for controlling the vehicle-mounted device 20. The RAM 20C may temporarily store programs or data as a working area.

[0031] The in-vehicle communication I / F 20D may be interpreted as an interface for connecting to the ECU 22. The interface may use a communication standard based on the CAN protocol. The in-vehicle communication I / F 20D is connected to the external bus 20H. The wireless communication I / F 20E may be interpreted as a wireless communication module for communicating with the center server 30. The wireless communication module may use a communication standard such as 5G, LTE, or Wi-Fi (registered trademark). The wireless communication I / F 20E is connected to the network N1. The input / output I / F 20F may be interpreted as an interface for communicating with the distance sensor 25 and the GPS device 29 mounted on the moving object 11.

[0032] The ECU 22 may be interpreted as an electronic control unit that controls the entire moving body 11. A plurality of ECUs 22 are provided for each function of the moving body 11. Specifically, an ECU 22 may be provided for each of the electric motors, brakes, cameras, etc. of the moving body 11. An in-vehicle device 23 is connected to each ECU 22. The in-vehicle device 23 is a device for realizing the functions of the moving body 11. Specifically, a brake actuator, which is the in-vehicle device 23, may be connected to the ECU 22 for the brake.

[0033] The distance sensor 25 may be interpreted as a device that measures distance. Specifically, the distance sensor 25 may be interpreted as a device that generates distance information indicating the distance from an object around the mobile body 11 to the mobile body 11. More specifically, the distance sensor 25 may be interpreted as LiDAR (Light Detection And Ranging). The LiDAR may be interpreted as a sensor that irradiates laser light and measures the distance to an object, the shape of the object, and the like based on information on the reflected light. The distance information may be transmitted to the center server 30 shown in FIG. 1 via the input / output I / F 20F. The distance sensor 25 may include a sensor that can calculate distance information, such as a stereo camera.

[0034] The GPS device 29 is a device that measures the current position of the mobile unit 11. The GPS device 29 may include an antenna (not shown) that receives signals from GPS satellites.

[0035] Next, the hardware configuration of the center server 30 will be described with reference to Fig. 3. Fig. 3 is a diagram showing the hardware configuration of the center server. The center server 30 includes a CPU 30A, a ROM 30B, a RAM 30C, a storage 30D, and a communication I / F 30E. The CPU 30A, the ROM 30B, the RAM 30C, the storage 30D, and the communication I / F 30E are connected to each other via an internal bus 30G so as to be able to communicate with each other. The functions of the CPU 30A, the ROM 30B, the RAM 30C, and the communication I / F 30E may be interpreted as being equivalent to the functions of the CPU 20A, the ROM 20B, the RAM 20C, and the wireless communication I / F 20E of the vehicle-mounted device 20 described above.

[0036] The storage 30D may include a hard disk drive (HDD), a solid state drive (SSD), etc. The storage 30D may store various programs and various data. The CPU 30A may read a program from the storage 30D and execute the program using the RAM 30C as a working area.

[0037] The storage 30D may store a blind spot area extraction program 100. The CPU 30A reads the blind spot area extraction program 100 from the storage 30D, expands the blind spot area extraction program 100 in the RAM 30C, and executes the expanded blind spot area extraction program 100.

[0038] Next, the function of the blind spot area extraction device realized by the CPU 30A of the center server 30 will be described with reference to Figs. 4 to 13. Fig. 4 is a block diagram showing the function of the center server. Fig. 5 is a diagram for explaining a blind spot area when an object around a moving body is seen from a stationary moving body. Fig. 6 is a diagram for explaining a blind spot area when an object around a moving body is seen from a moving moving body. Fig. 7 is a diagram showing an example of an occupancy grid map. Fig. 8 is a diagram for explaining a state where a change history of the occupancy grid map is accumulated in a time series change map. Fig. 9 is a diagram for explaining a state where a change history of the occupancy grid map is accumulated in a time series change map. Fig. 10 is a diagram for explaining a state of similarity evaluation. Fig. 11 is a diagram for explaining an example of a window W having a characteristic pattern. Fig. 12 is a diagram showing a result of similarity evaluation. Fig. 13 is a diagram showing an example of a pattern.

[0039] The CPU 30A includes a distance information acquisition unit 31, a map data storage unit 32, and a pattern extraction unit 33. The distance information acquisition unit 31, the map data storage unit 32, and the pattern extraction unit 33 may be realized by the CPU 30A executing a blind spot area extraction program 100 (FIG. 3).

[0040] (distance information acquisition section) The distance information acquisition unit 31 may acquire distance information DIS from the distance sensor 25, as shown in Fig. 5. Specifically, the distance information acquisition unit 31 may acquire distance information DIS indicating a plurality of specific distances to an object 50 around the moving body 11. The object 50 may include a building, a utility pole, a guardrail, a wall, a vehicle, a person, and the like. The specific direction D1 shown in Fig. 5 may be interpreted as the traveling direction D of the moving body 11. The specific direction D2 intersecting the specific direction D1 may be interpreted as a direction perpendicular to the traveling direction D of the moving body 11.

[0041] 6, when the moving object 11 is moving in the traveling direction D, the distance information acquisition unit 31 may acquire a plurality of pieces of distance information DIS transmitted in chronological order from the distance sensor 25 as the moving object 11 moves. Specifically, the distance information acquisition unit 31 may obtain the pieces of distance information DIS acquired by the distance sensor 25 at positions P1, P2, and P3 of the moving object 11 in the traveling direction D.

[0042] Position P1 may be interpreted as a first specific position of the moving body 11 in the traveling direction D. Position P2 may be interpreted as a second specific position of the moving body 11 that is a certain distance away from position P1 in the traveling direction D. Position P3 may be interpreted as a third specific position of the moving body 11 that is a certain distance away from position P2 in the traveling direction D.

[0043] (Map data storage section) The map data storage unit 32 may store a plurality of map data based on the plurality of pieces of distance information DIS acquired by the distance information acquisition unit 31.

[0044] Specifically, the map data storage unit 32 may generate multiple map data including blind spot areas caused by the object 50 when the object 50 is viewed from a moving body 11 located at a specific position, based on multiple distance information DIS, and store the map data in chronological order.

[0045] More specifically, the map data storage unit 32 may store first map data including at least a first blind spot area BD1 that is a blind spot caused by the object 50 when the object 50 is viewed from the moving object 11 that is located at a position P1 that is a first position. The map data storage unit 32 may also store second map data including a second blind spot area BD2 that is a blind spot caused by the object 50 when the object 50 is viewed from the moving object 11 that is located at a position P2 that is a second specific position different from the first specific position.

[0046] The map data storage unit 32 may store, in addition to the map data (first map data and second map data) including two different blind spot areas, third map data including another blind spot area, for example, the third blind spot area BD3 shown in FIG. 6, and may further store, in chronological order, fourth map data including another blind spot area.

[0047] (Example of map generation method) The map data storage unit 32 stores a plurality of pieces of distance information by the method disclosed in Non-Patent Document 1. Specifically, the map data storage unit 32 uses the distance sensor 25 mounted on the mobile unit 11 to scan the surroundings of the mobile unit 11 by 360[deg]. The distance sensor 25 irradiates one or more objects 50 around the mobile unit 11 with laser light and receives the laser light reflected by the object 50, and accumulates a plurality of pieces of distance information DIS (FIG. 5) from the object 50 to the mobile unit 11 by utilizing the time from the irradiation time of the laser light to the reception time. The distance sensor 25 repeats acquiring the distance information DIS a plurality of times, and the map data storage unit 32 may determine an area where the object 50 exists (grid), an area where the object 50 does not exist (free), an area where it is uncertain whether the object 50 exists (Unknown), and the like, and generate a map including these areas. The grid may be interpreted as a cell range where the object 50 exists.

[0048] When generating a map including these regions, the map data storage unit 32 may generate an occupancy grid map by the method disclosed in Non-Patent Document 2 or Non-Patent Document 3, and may generate a map including regions such as grid, free, and unknown by using the occupancy grid map to calculate the occupancy probability around the moving body 11. For example, as shown in FIG. 7, the grid (cell) region between two points (start point / end point) through which the laser light LD passes may be changed to a free area FA. The free area FA may be interpreted as the shaded cell in FIG. 7. In order to improve the accuracy of the determination, the occupancy probability (odds) disclosed in Non-Patent Document 3 may be used to determine the free area FA from the unknown area UA based on multiple scan information.

[0049] The map data storage unit 32 may store the change history of the occupancy grid map in a time-series change map when the moving object 11 moves from the start point PS toward the goal point PG in the specific map shown in Fig. 8. Specifically, (1) when the moving object 11 moves, the map data storage unit 32 stores the distance information DIS obtained by the distance sensor 25 scanning 360 [deg]. (2) As shown in Fig. 9, the map data storage unit 32 may store the change history of the occupancy grid map until the moving object 11 reaches the goal point PG in a time-series change map. The time-series change map may be interpreted as a motion template disclosed in Non-Patent Document 5.

[0050] The map data accumulation unit 32 may evaluate the similarity of the images while shifting a window W having a characteristic pattern on the time-series change map as shown in FIG. 10. The window W may be interpreted as a region of interest. The characteristic pattern may be interpreted as a fan-shaped gradation pattern as shown in FIG. 11. FIG. 11 shows an example of the size of the window W having the gradation pattern. The map data accumulation unit 32 may evaluate the similarity multiple times while changing the size of the window W. The LBP disclosed in Non-Patent Document 4 may be used to evaluate the similarity. By evaluating the similarity, it is possible to extract a blind spot area BD of the object 50 as shown in FIG. 12.

[0051] (Pattern Extraction Section) The pattern extraction unit 33 may extract the pattern PI shown in Fig. 13 based on at least the first map data and the second map data. Specifically, the pattern extraction unit 33 may use the time-series change map (e.g., a motion template) described above to extract the pattern PI, which is an image showing the time-series change of the first blind spot area BD1, the second blind spot area BD2, and the third blind spot area BD3. The pattern PI may be interpreted as an image showing the areas with different color densities in the first blind spot area BD1, the second blind spot area BD2, and the third blind spot area BD3. The pattern PI may be interpreted as an image showing the areas with different patterns in the first blind spot area BD1, the second blind spot area BD2, and the third blind spot area BD3.

[0052] Next, the operation of the blind spot area extraction system will be described with reference to Fig. 14. Fig. 14 is a flow chart for explaining the operation of the blind spot area extraction system.

[0053] When the blind spot area extraction system 10 is started in step S1, the blind spot area extraction device acquires distance information DIS from the distance sensor 25 in step S2.

[0054] When the blind spot area extraction device updates the occupied grid map in step S3, the blind spot area extraction device determines in step S4 whether the updating of the occupied grid map has been completed.

[0055] If the update is not completed (step S4: NO), the blind spot area extraction device repeats the process from step S2 onwards. If the update is completed (step S4: YES), in step S5, the blind spot area extraction device calculates the difference from the occupancy grid map n time points before.

[0056] In step S6, the blind spot area extraction device accumulates the difference information in the time-series change map, and in step S7, the blind spot area extraction device determines whether accumulation of the difference information has been completed.

[0057] If the accumulation is not completed (step S7: NO), the blind spot area extraction device repeats the processes from step S2 to step S8 by moving the distance sensor 25 in step S8 until the accumulation of the difference information is completed.

[0058] If the accumulation is completed (step S7: YES), in step S9, the blind spot area extraction device extracts the features of the blind spot area and similar areas. In step S10, the blind spot area extraction device determines whether the evaluation of the image similarity is completed or not, and if the evaluation is not completed (step S10: NO), the process from step S9 onwards is repeated. If the evaluation is completed (step S10: YES), the series of processes is completed.

[0059] (Other Examples) Depending on the moving path of the moving body 11, there may be a blind spot area that is difficult to see unless the moving body 11 moves close to the object 50. Also, there may be an area that is unlikely to become a blind spot if it is a position relatively far from the object 50. This will be described with reference to FIG. 15. FIG. 15 is a diagram for explaining the function of determining the depth of the blind spot area. The upper side of FIG. 15 shows a pattern extracted when the moving body 11 moves close to the object 50. That is, the upper side of FIG. 15 shows a blind spot area that may occur when the moving body 11 moves close to the object 50. The lower side of FIG. 15 shows a pattern extracted when the moving body 11 moves in a place relatively far from the object 50. That is, the lower side of FIG. 15 shows a blind spot area that may occur when the moving body 11 moves in a place relatively far from the object 50.

[0060] In this way, the depth of the blind spot area may vary depending on the moving path of the moving body 11. The depth of the blind spot area may be interpreted as the degree of depth of the blind spot area, or may be interpreted as the degree of width of the blind spot area. The depth of the blind spot area may be interpreted as the degree of width of the blind spot area in the traveling direction of the moving body 11, or may be interpreted as the degree of width of the blind spot area in a direction perpendicular to the traveling direction of the moving body 11. The depth of the blind spot area may be interpreted as the shape and size of the blind spot area.

[0061] Therefore, the above-mentioned pattern may differ depending on the moving path of the moving object 11. In other words, the interval of the fan-shaped gradation pattern shown in FIG. 11 may change. By determining such a difference in pattern, it is possible to determine the depth of the blind spot area caused by the difference in the moving path of the moving object 11. Then, by determining the depth of the blind spot, it may be possible to select, from among a plurality of moving paths, a moving path that is likely to have a shallower blind spot as seen from the moving object 11, depending on the depth of the blind spot.

[0062] The pattern extraction unit 33 of the blind spot extraction device according to another embodiment may extract the first blind spot area and the second blind spot area, etc., included in the above-mentioned pattern, determine the depth of the first blind spot area and the second blind spot area based on the respective shapes of the extracted first blind spot area and the second blind spot area, etc., and set a parameter corresponding to the determined depth.

[0063] Next, the operation of the blind spot area extraction system according to another embodiment will be described with reference to Fig. 16. Fig. 16 is a flow chart for explaining another operation of the blind spot area extraction system. Fig. 16 is a diagram for explaining another operation of the blind spot area extraction system. The processing from step S1 to step S10 shown in Fig. 16 is the same as the processing shown in Fig. 14, so the description will be omitted.

[0064] After completing the evaluation of the similarity of the images in step S10, the blind spot area extraction device determines the depth of the blind spot in step S12 if the blind spot depth evaluation is to be performed (step S11: YES). Specifically, the depth of the blind spot may be determined by using the characteristics of each of the blind spots.

[0065] More specifically, the blind spot area extraction device extracts the corners of the multiple first blind spot areas BD1 detected by the moving object 11 moving along different moving paths by thinning processing, as shown in FIG. 15, and obtains the angle θ1 of each corner. Similarly, the blind spot area extraction device obtains the angle θ2 of each of the multiple second blind spot areas BD1. Then, the blind spot area extraction device compares the angle θ1 and the angle θ2 with the specific reference angle θ. Specifically, the blind spot area extraction device may compare the angle θ1 and the angle specific reference angle θ, and may compare the angle θ2 and the angle specific reference angle θ. Furthermore, the blind spot area extraction device may determine the depth of the blind spot based on the magnitude of each angle difference, and finally set a parameter corresponding to the determined depth.

[0066] After determining the depth of the blind spot area, in step S13, the blind spot area extraction device determines whether the determination of the depth of the blind spot area is complete. If the determination of the depth of the blind spot area is not complete (step S13: NO), the blind spot area extraction device repeats the process from step S12 onwards. If the determination of the depth of the blind spot area is complete (step S13: YES), the series of processes is completed.

[0067] As described above, the blind spot area extraction device according to an embodiment of the present disclosure accumulates the first map data including the first blind spot area and the second map data including the second blind spot area in chronological order, and extracts a pattern, which is an image showing changes over time in the first blind spot area and the second blind spot area, based on the accumulated first map data and second map data.

[0068] With this configuration, a travel route can be planned based on the patterns of blind spots that may occur when the moving body 11 passes through multiple travel routes, so that the moving body 11 passes through areas with as shallow a blind spot as possible.

[0069] In addition, depending on the depth of the blind spot area, it is possible to appropriately control the speed of the moving body 11 and control the timing of the generation of a warning sound from the moving body 11, taking into account the degree of danger to the moving body 11, that is, the degree to which obstacles such as people and vehicles present in the blind spot area may come into contact with the moving body 11.

[0070] The processor of the present disclosure may extract the pattern by using a feature of a local binary pattern (LBP) calculated based on the pixel values ​​of a specific pixel included in the pattern, which is an image showing the time-series changes in the first blind spot area and the second blind spot area, and the surrounding pixels around the specific pixel. By using the feature of the LBP, the pattern can be extracted with high accuracy without being affected by the change in illuminance around the moving object.

[0071] The processor of the present disclosure may extract a pattern by regarding a pattern, which is an image showing a time-series change in the first blind spot area and the second blind spot area based on BoVW (Bag of Visual Words), as a collection of local features, converting the local features into a histogram, and searching the histogram by machine learning. By searching the histogram of the local features in the images of the first blind spot area and the second blind spot area, a pattern can be extracted with high accuracy.

[0072] In the above embodiment, various processes executed by the CPU 30A by reading the software (program) may be executed by various processors other than the CPU. In this case, examples of the processor include a PLD (Programmable Logic Device) such as an FPGA (Field-Programmable Gate Array) whose circuit configuration can be changed after manufacture, and a dedicated electric circuit such as an ASIC (Application Specific Integrated Circuit) which is a processor having a circuit configuration designed exclusively for executing a specific process. The above-mentioned reception process may be executed by one of these various processors, or may be executed by a combination of two or more processors of the same or different types (for example, a plurality of FPGAs, and a combination of a CPU and an FPGA). The hardware structure of these various processors is, more specifically, an electric circuit that combines circuit elements such as semiconductor elements.

[0073] The above-mentioned program may be provided in a form recorded on a non-transitory recording medium such as a CD-ROM (Compact Disc Read Only Memory), a DVD-ROM (Digital Versatile Disc Read Only Memory), a USB (Universal Serial Bus) memory, etc. The program may be provided in a form downloaded from an external device via a network.

[0074] The processes in the above embodiments may be executed not only by one processor but also by multiple processors working together. The process flow described in the above embodiments is also an example, and unnecessary steps may be deleted, new steps may be added, or the order of processes may be changed without departing from the spirit of the invention.

[0075] Furthermore, the present invention is not limited to the above, and it goes without saying that various modifications can be made without departing from the spirit and scope of the present invention.

[0076] (Additional Note) The following supplementary notes are further disclosed regarding the above-described embodiment.

[0077] (Appendix 1) A blind spot extraction device including at least one processor, The processor, acquiring a plurality of pieces of distance information indicating distances from a distance sensor mounted on a moving body to objects around the moving body in chronological order from the distance sensor as the moving body moves; generating first map data including at least a first blind spot area caused by the object when the object is viewed from the moving body located at a first specific position, and second map data including a second blind spot area caused by the object when the object is viewed from the moving body located at a second position different from the first specific position, based on the plurality of pieces of distance information, and storing the data in chronological order; A pattern, which is an image showing a time-series change in the first blind spot area and the second blind spot area, is extracted based on the first map data and the second map data.

[0078] (Appendix 2) The blind spot extraction device according to claim 1, wherein the processor extracts the first blind spot area and the second blind spot area included in the pattern, and sets parameters corresponding to depths of the first blind spot area and the second blind spot area based on respective shapes of the extracted first blind spot area and the second blind spot area.

[0079] (Appendix 3) 3. The blind spot extraction device according to claim 1, wherein the pattern is an image showing areas of the first blind spot area and the second blind spot area having different color intensities.

[0080] (Appendix 4) The blind spot extraction device according to any one of appendices 1 to 3, wherein the processor extracts the pattern by utilizing features of a Local Binary Pattern (LBP) calculated based on pixel values ​​of a specific pixel included in the pattern, which is an image showing a time-series change in the first blind spot area and the second blind spot area, and surrounding pixels around the specific pixel.

[0081] (Appendix 5) The blind spot extraction device according to any one of appendices 1 to 4, wherein the processor extracts the pattern by regarding a pattern, which is an image showing a time-series change in the first blind spot area and the second blind spot area, as a collection of local features based on BoVW (Bag of Visual Words), converting the local features into a histogram, and searching the histogram by machine learning.

[0082] (Appendix 6) The blind spot area extraction device according to any one of appendixes 1 to 5, wherein the distance sensor includes a LiDAR (Light Detection And Ranging).

[0083] (Appendix 7) acquiring a plurality of pieces of distance information indicating distances from a distance sensor mounted on a moving body to objects around the moving body in chronological order from the distance sensor as the moving body moves; accumulating, in chronological order, first map data including a first blind spot area that is a blind spot caused by the object when the object is viewed from the moving body that is located at a first specific position, and second map data including a second blind spot area that is a blind spot caused by the object when the object is viewed from the moving body that is located at a second position different from the first specific position, based on the plurality of pieces of distance information; extracting a pattern, which is an image showing a time-series change in the first blind spot area and the second blind spot area, based on the first map data and the second map data; This is a blind spot extraction method performed by a computer.

[0084] (Appendix 8) acquiring a plurality of pieces of distance information indicating distances from a distance sensor mounted on a moving body to objects around the moving body in chronological order from the distance sensor as the moving body moves; accumulating, in chronological order, first map data including a first blind spot area that is a blind spot caused by the object when the object is viewed from the moving body that is located at a first specific position, and second map data including a second blind spot area that is a blind spot caused by the object when the object is viewed from the moving body that is located at a second position different from the first specific position, based on the plurality of pieces of distance information; extracting a pattern, which is an image showing a time-series change in the first blind spot area and the second blind spot area, based on the first map data and the second map data; A blind spot extraction program that enables a computer to do this. [Explanation of symbols]

[0085] 10 Blind Spot Extraction System 11 Mobile 20 Onboard equipment 20A CPU 20B ROM 20C RAM 20D In-vehicle communication I / F 20F input / output I / F 20G Internal Bus 20H External Bus 22 ECU 23 Onboard equipment 25 Distance Sensor 29 GPS device 30 Center Server 30A CPU 30B ROM 30C RAM 30D Storage 30E Communication I / F 30G Internal Bus 31 Distance information acquisition section 32 Map data storage unit 33 Pattern Extraction Section 40 Information Server 50 objects 100 Blind Spot Extraction Program

Claims

1. A blind spot extraction device including at least one processor, The processor, acquiring a plurality of pieces of distance information indicating distances from a distance sensor mounted on a moving body to objects around the moving body in chronological order from the distance sensor as the moving body moves; generating first map data including at least a first blind spot area caused by the object when the object is viewed from the moving body located at a first specific position, and second map data including a second blind spot area caused by the object when the object is viewed from the moving body located at a second position different from the first specific position, based on the plurality of pieces of distance information, and storing the data in chronological order; A blind spot area extraction device that extracts a pattern, which is an image showing a time-series change in the first blind spot area and the second blind spot area, based on the first map data and the second map data.

2. 2. The blind spot extraction device according to claim 1, wherein the processor extracts the first blind spot area and the second blind spot area included in the pattern, and sets parameters corresponding to depths of the first blind spot area and the second blind spot area based on respective shapes of the extracted first blind spot area and the extracted second blind spot area.

3. The blind spot extraction device according to claim 1 , wherein the pattern is an image showing the first blind spot area and the second blind spot area having different color intensities.

4. 2. The blind spot extraction device according to claim 1, wherein the processor extracts the pattern by using a feature amount of a local binary pattern (LBP) calculated based on pixel values ​​of a specific pixel included in the pattern, which is an image showing a time-series change in the first blind spot area and the second blind spot area, and surrounding pixels around the specific pixel.

5. 2. The blind spot area extraction device according to claim 1, wherein the processor extracts the pattern by regarding a pattern, which is an image showing a time-series change in the first blind spot area and the second blind spot area, as a collection of local features based on BoVW (Bag of Visual Words), converting the local features into a histogram, and searching the histogram by machine learning.

6. The blind spot area extraction device according to claim 1 , wherein the distance sensor includes a LiDAR (Light Detection And Ranging) sensor.

7. acquiring a plurality of pieces of distance information indicating distances from a distance sensor mounted on a moving body to objects around the moving body in chronological order from the distance sensor as the moving body moves; accumulating, in chronological order, first map data including a first blind spot area that is a blind spot caused by the object when the object is viewed from the moving body that is located at a first specific position, and second map data including a second blind spot area that is a blind spot caused by the object when the object is viewed from the moving body that is located at a second position different from the first specific position, based on the plurality of pieces of distance information; extracting a pattern, which is an image showing a time-series change in the first blind spot area and the second blind spot area, based on the first map data and the second map data; This is a blind spot extraction method performed by a computer.

8. acquiring a plurality of pieces of distance information indicating distances from a distance sensor mounted on a moving body to objects around the moving body in chronological order from the distance sensor as the moving body moves; accumulating, in chronological order, first map data including a first blind spot area that is a blind spot caused by the object when the object is viewed from the moving body that is located at a first specific position, and second map data including a second blind spot area that is a blind spot caused by the object when the object is viewed from the moving body that is located at a second position different from the first specific position, based on the plurality of pieces of distance information; extracting a pattern, which is an image showing a time-series change in the first blind spot area and the second blind spot area, based on the first map data and the second map data; A blind spot extraction program that enables a computer to do this.

Images