Information generation method, information generation device, and program

Abstract

This information-generating method according to the present invention is in an information-generating device for generating information for machine learning to estimate whether or not it is possible for a mobile body (100) to move in a prescribed region. When the mobile body (100) moves in a first region, (1) first information acquired from at least a sensor arranged in the mobile body (100), and (2) second information relating to movement of the mobile body (100) are acquired, and according to the second information, an estimate is made of whether the mobile body (100) can move in the first region, and fourth information for a learning model is generated in which the first information and the second information are associated with third information indicating the result of estimating whether movement is possible.

Description

[Technical Field]

[0001] The present invention relates to an information generation method, an information generation device, and a program. [Background technology]

[0002] In recent years, machine learning has increasingly been used to control robots. For example, a technology has been disclosed that uses machine learning to automatically control a robot in a manner similar to that of a human operator (see Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2018-149669 Summary of the Invention [Problem to be solved by the invention]

[0004] Incidentally, one of the challenges in machine learning is that it is known to be difficult to generate appropriate training data.

[0005] The present disclosure provides an information generation method and the like that can appropriately generate learning data. [Means for solving the problem]

[0006] An information generation method according to one aspect of the present invention estimates whether a moving object can move through a predetermined area. Learning ModelAn information generation method in an information generation device for generating information for a learning model, wherein when a moving object moves through a first area, (1) first information acquired from at least a sensor installed on the moving object, and (2) second information regarding the movement of the moving object are acquired, and whether or not the moving object is capable of moving through the first area is estimated based on the second information, and fourth information for the learning model is generated by associating the first information and the second information with third information indicating the result of the estimation of whether or not the moving object is capable of moving.

[0007] These comprehensive or specific aspects may be realized as a system, device, integrated circuit, computer program, or computer-readable recording medium such as a CD-ROM, or may be realized as any combination of a system, device, integrated circuit, computer program, and recording medium. [Effects of the Invention]

[0008] The information generation method of the present disclosure can appropriately generate learning data. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram schematically illustrating the configuration of a learning system according to an embodiment. [Figure 2] FIG. 2 is a diagram schematically illustrating a functional configuration of a vehicle according to an embodiment. [Figure 3] FIG. 3 is a diagram schematically illustrating the configuration of a remote control device according to an embodiment. [Figure 4] FIG. 4 is a diagram illustrating an example of semi-automatic remote control according to the embodiment. [Figure 5] FIG. 5 is a first diagram illustrating an example of training data according to the embodiment. [Figure 6] FIG. 6 is a second diagram illustrating an example of the training data according to the embodiment. [Figure 7] FIG. 7 is a flowchart showing the processing of the learning system according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] (Knowledge that served as the basis for disclosure) The inventors of the present disclosure have found that the following problems arise with the technology described in the "Background Art" section for automatically controlling (autonomously operating) robots and the like using machine learning.

[0011] In order to automatically perform such control that mimics human operation, it is necessary to learn control patterns using a huge amount of training data. Each piece of this huge amount of training data is annotated with annotation information such as correct / incorrect to distinguish between control that is a correct control pattern and control that is an incorrect control pattern. In the control of a robot or the like, it is sufficient to add simple annotation information such as "move" or "don't move," so adding such annotation information can be done relatively easily.

[0012] However, in order to automatically determine a movement area (also referred to as a driving area) for automatically moving a mobile object, for example, it is necessary to add annotation information indicating movable and immovable areas to an image of a specific area. Adding such annotation information requires a person to visually view the image and specify the movable and immovable areas pixel by pixel, which requires a high human resource cost. This human resource cost makes it difficult to generate a large amount of data for learning. Therefore, the present disclosure provides an information generation device capable of automatically adding annotation information to a self-propelled (autonomously mobile) mobile object and generating information for machine learning, as well as an information generation method for the information generation device.

[0013] This information generation method makes it possible to generate training data that can be used to train a learning model for determining the movement area of ​​a self-propelled mobile body while keeping human costs low. This makes it possible to accelerate the preparation of training data, which has traditionally been a barrier to updating the learning model for determining the movement area, and enables frequent updates of the learning model.

[0014] In order to solve the above problem, an information generation method according to an aspect of the present disclosure includes: Learning Model An information generation method in an information generation device for generating information for a mobile object, which, when the mobile object moves through a first area, acquires (1) first information acquired from at least a sensor installed on the mobile object, and (2) second information regarding the movement of the mobile object, estimates whether the mobile object can move through the first area according to the second information, and generates fourth information for a learning model that corresponds the first information and the second information to third information indicating the result of the estimation of whether the mobile object can move.

[0015] This information generation method generates fourth information that associates first information regarding the movement of a mobile object when the mobile object moves within a first area, second information acquired from a sensor when the mobile object moves within the area, and third information, which is an estimation result of whether the mobile object can move through the first area. The fourth information, that is, the result of estimating whether the mobile object can move through the first area, is annotated. By performing machine learning using the fourth information, a learning model can be constructed that can output, from the first information and second information about a specific area, whether the mobile object can move through the specific area corresponding to the third information. In this way, because the fourth information to which annotation information is automatically annotated is generated, there is no need to perform the conventional annotation operation, which requires a high labor cost. Therefore, learning data can be appropriately generated.

[0016] Also, for example, an estimation of whether a moving body is capable of moving through a first area may be based on whether the difference between the movement data during movement through the first area contained in the first information obtained when the moving body moves in accordance with the second information and a threshold value is within a predetermined range.

[0017] According to this, the third information can be generated based on the first information obtained when the moving object moves according to the second information. As a result, the fourth information that associates the first information and the second information with the third information can be generated, so that the annotation operation that requires a large human cost as in the past is not necessary. Therefore, the learning data can be generated appropriately.

[0018] Also, for example, the second information may be input by an operator who remotely controls the mobile object.

[0019] This allows the fourth information, i.e., learning data, to be appropriately generated from the first information and the second information when the mobile object moves in accordance with the second information input by an operator remotely controlling the mobile object.

[0020] Furthermore, for example, the difficulty of the movement of the moving body may be further estimated, and in generating the fourth information, the fourth information may be generated by associating the first information and the second information with fifth information indicating the result of the estimation of the difficulty and the third information.

[0021] This allows for the generation of fourth information including the fifth information associated with the first information, the second information, and the third information. In the generated fourth information, the manner of machine learning, etc., can be changed based on the associated fifth information in accordance with the estimation result of the difficulty of movement of the moving object.

[0022] Furthermore, for example, the fourth information may be generated only when the third information and the fifth information satisfy a predetermined condition.

[0023] According to this, the fourth information is generated from the third information and the fifth information only when a predetermined condition indicating that the generated fourth information is appropriate learning data is met, and the fourth information is not generated when the predetermined condition indicating that the generated fourth information is not appropriate learning data is not met. As a result, it is possible to avoid generating fourth information that is not appropriate for use as learning data and therefore meaningless to generate.

[0024] Furthermore, for example, the fourth information may include a reliability estimated based on the first information.

[0025] According to this, fourth information including reliability estimated based on the first information is generated. By using this fourth information as learning data, the reliability information can be used during machine learning.

[0026] Furthermore, for example, sixth information may be acquired that divides the first area into multiple sections by type, and the fourth information may include a composite image in which information indicating whether a moving body based on the third information is movable is superimposed on the image of the first area for each of the multiple sections obtained when the image of the first area included in the first information is divided by the sixth information.

[0027] This makes it possible to output a composite image in which information as to whether the moving object is movable or not is superimposed in each of a plurality of sections obtained by dividing one image based on the sixth information.

[0028] Furthermore, for example, based on the first information, other moving bodies present in the first area may be identified, and when estimating whether or not the first area is movable, at least one of (a) estimating that the first area in which the other moving bodies are present is movable if the other moving bodies satisfy a first condition, and (b) estimating that the first area in which the other moving bodies are present is not movable if the other moving bodies satisfy a second condition may be performed.

[0029] According to this, when another moving object in the first area satisfies the first condition, it is possible to generate third information that estimates that the first area in which the other moving object exists is movable, and when another moving object in the first area satisfies the second condition, it is possible to generate fourth information that estimates that the other moving object in the first area is not movable. Since such third information can be generated as fourth information that is associated with the first information and the second information, there is no need to perform the annotation operation that requires a high human resource cost as in the past. Therefore, it is possible to appropriately generate learning data.

[0030] Furthermore, a program according to one aspect of the present disclosure is a program for causing a computer to execute the information generation method described above.

[0031] This makes it possible to achieve the same effects as the information generating method described above using a computer.

[0032] In addition, an information generation device according to one embodiment of the present disclosure is an information generation device for generating information for a learning model that estimates whether a moving body is capable of moving within a predetermined area, and includes: an acquisition unit that acquires (1) first information acquired from at least a sensor installed on the moving body and (2) second information regarding the first area when the moving body moves within a first area; an estimation unit that estimates whether the moving body is capable of moving within the first area based on the first information; and a generation unit that generates fourth information for the learning model that associates the first information and the second information with third information indicating the result of the estimation of whether the moving body is capable of moving.

[0033] This can achieve the same effects as the information generation method described above.

[0034] These comprehensive or specific aspects may be realized as a system, device, integrated circuit, computer program, or computer-readable recording medium such as a CD-ROM, or may be realized as any combination of a system, device, integrated circuit, computer program, or recording medium.

[0035] Hereinafter, the embodiments will be specifically described with reference to the drawings.

[0036] The embodiments described below are all comprehensive or specific examples. The numerical values, shapes, materials, components, component placement and connection configurations, steps, and step order shown in the following embodiments are merely examples and are not intended to limit the present invention. Furthermore, among the components in the following embodiments, components that are not described in the independent claims that represent the highest concepts are described as optional components.

[0037] (Embodiment) First, the configuration of the learning system in this embodiment will be described.

[0038] Fig. 1 is a diagram showing a schematic configuration of a learning system according to an embodiment, and Fig. 2 is a diagram showing a schematic functional configuration of a vehicle according to an embodiment.

[0039] As shown in FIG. 1, a learning system 500 of this embodiment is realized by a mobile object 100, a server device 200 connected via a network 150, and a remote control device 300.

[0040] The mobile object 100 is a device capable of autonomous movement, such as an automobile, robot, drone, bicycle, or wheelchair, and is used, for example, to deliver luggage or the like while it is loaded with the luggage. The mobile object 100 includes a motor or the like as a power source, a drive unit such as wheels driven by the motor, and a functional unit that stores a power source (e.g., electricity) for operating the motor. Note that the learning system 500 only needs to be able to generate annotated training data, and a mobile object dedicated to this learning may be used. This mobile object dedicated to learning may not include functions such as loading luggage, or may not include functions for autonomous movement.

[0041] Network 150 is a communication network for connecting mobile object 100, server device 200, and remote control device 300 so that they can communicate with each other. Here, a communication network such as the Internet is used as network 150, but is not limited to this. Furthermore, the connection between mobile object 100 and network 150, the connection between server device 200 and network 150, and the connection between remote control device 300 and network 150 may be made by wireless communication or by wired communication. However, due to the nature of mobile object 100, it is preferable that mobile object 100 be connected to network 150 by wireless communication.

[0042] To connect the mobile object 100 to the network 150 via wired communication, it is sufficient that the mobile object 100 is equipped with a storage device for storing various data. Then, when it is time to replenish the power source of the mobile object 100, a wired connection with the network 150 is formed and the various data stored in the storage device is transmitted to the network 150.

[0043] 2, the mobile object 100 includes a control unit 101, a control information receiving unit 102, a sensor information transmitting unit 103, a sensing unit 104, and a recognition unit 105. Each of these functional blocks constituting the mobile object 100 is realized using, for example, a processor and a memory. Each functional block will be described in detail below.

[0044] The server device 200 is a device for performing information processing and the like, and is realized using, for example, a processor and a memory. The server device 200 may be realized by an edge computer or a cloud computer. Furthermore, one server device 200 may be provided for one mobile object 100, or one server device 200 may be provided for multiple mobile objects 100.

[0045] 2, the server device 200 includes a remote control unit 201, a control information transmission unit 202, a sensor information reception unit 203, a learning data generation unit 204, a learning unit 205, and a storage device 206. Details of each of the functional blocks constituting the server device 200 will be described later.

[0046] FIG. 3 is a diagram schematically illustrating the configuration of a remote control device according to an embodiment. The remote control device 300 is realized by, for example, a computer. As shown in FIG. 3, the remote control device 300 includes, for example, a display device 301 and a steering unit 302, which is a user interface for input. In addition to the steering unit 302, the user interface may include a keyboard, a touch panel, a mouse, a dedicated controller, a foot panel, a combination of VR glasses and a controller, a sound collector for voice input, or a combination of two or more of these. The remote control device 300 is used to remotely control the moving object 100.

[0047] Continuing with reference to FIGS. 1 to 3, the transmission and reception of data between the components will be described below, along with the function of each component.

[0048] The sensing unit 104 is connected to sensors (not shown) and acquires the results of sensing the environment of the mobile object 100 (sensor information: first information) from these sensors. These sensors include, for example, a camera, LiDAR, radar, sonar, a microphone, GPS, a vibration sensor, an acceleration sensor, a gyro sensor, a temperature sensor, etc. The sensing unit 104 transmits the acquired sensor information to the sensor information transmission unit 103, the recognition unit 105, and the control unit 101.

[0049] The recognition unit 105 acquires the sensor information transmitted from the sensing unit 104, recognizes the environment of the mobile object 100, and transmits the recognition result to the sensor information transmission unit 103 and the control unit 101. The recognition of the environment performed by the recognition unit 105 includes generating information (sixth information) that divides the environment of the mobile object 100 into multiple sections by type. In addition, the recognition unit 105 recognizes the environment necessary for the movement of the mobile object 100. For example, the recognition unit 105 recognizes obstacles to the mobile object 100, other mobile objects, and areas in which the mobile object 100 can move.

[0050] The recognition unit 105 also performs a process of recognizing the degree of difficulty of the movement of the moving body 100 by estimating the degree of difficulty. The degree of difficulty of the movement of the moving body 100 here is estimated based on, for example, the number of braking and turning required for the moving body 100 to move within an area, the magnitude and number of detections of vibrations and the like that adversely affect the movement of the moving body, and the number of other moving bodies, and the greater the number (or magnitude) of such vibrations, the greater the degree of difficulty of the movement of the moving body 100. Further examples of a condition where the degree of difficulty of the movement of the moving body 100 is high include narrow roads where it is difficult for the moving body 100 to pass other moving bodies, roads where cars are frequently parked on the road, roads where objects large enough to obstruct movement are scattered, and roads with a radio wave environment where communication quality with the network 150 is low (frequently interrupted).

[0051] At least a part of the environment recognition process may be executed sequentially as soon as sensor information is input from the sensing unit 104, or the sensor information may be buffered (accumulated) in a data storage area (not shown) and executed later (for example, after the delivery service using the mobile object 100 is completed). For the environment recognition here, a learning model trained in advance by machine learning may be used, or the process may be performed based on predetermined rules. This learning model may be trained using training data annotated with annotation information generated in this embodiment.

[0052] The sensor information transmitting unit 103 transmits the sensor information transmitted from the sensing unit 104 and the recognition result transmitted from the recognition unit 105 to the sensor information receiving unit 203 of the server device 200. Note that the transmission and reception of information between the mobile object 100 and the server device 200, including this communication, may be subject to information transformation such as compression for the purpose of data reduction and encryption for the purpose of maintaining confidentiality. In this case, the compressed information is decompressed and the encrypted information is decoded. Decryption The restoration of altered information, such as the above, may be performed in the functional block that received the altered information. Furthermore, various types of information may be transmitted and received only when the network bandwidth (communication capacity) between the mobile object 100 and the server device 200 is equal to or greater than a predetermined value. Furthermore, additional information such as an ID and a timestamp may be added to the transmitted and received information to ensure consistency. Thus, the form of information transmission and reception between the mobile object 100 and the server device 200 may be any form.

[0053] The sensor information receiving unit 203 transmits the sensor information and the recognition result transmitted from the sensor information transmitting unit 103 to the remote control unit 201 and the learning data generating unit 204 .

[0054] The remote control unit 201 determines the movement direction of the moving object 100 as needed based on the sensor information and recognition result transmitted from the sensor information receiving unit 203, and remotely controls the moving object 100 so that it moves in the determined movement direction. Specifically, control information (second information) related to the movement of the moving object 100 for remote control is generated and transmitted to the control information transmitting unit 202. The control information is also transmitted to the learning data generating unit 204.

[0055] In this embodiment, the decision of the moving direction of the moving body 100, that is, the remote control of the moving body 100, is performed by an operator. Therefore, the remote control device 300 shown in Fig. 3 is used by the operator.

[0056] For example, an image of the environment of the mobile object included in the sensor information is transmitted from the remote control unit 201 to the remote control device 300 and displayed on the display device 301. Instead of the display device 301, a display device such as a smartphone or VR / AR glasses may be used.

[0057] The operator checks the sensor information (here, an image of the environment) displayed on the display device 301, and if he determines that remote control is necessary, he can control the direction of movement of the mobile object 100. The operator determines the direction of movement by judging the area in which the mobile object 100 can move. The operator then provides input via a user interface such as the steering unit 302 so that the mobile object 100 moves in the determined direction of movement, and an input signal is transmitted from the remote control device 300 to the remote control unit 201 based on the input. The remote control unit 201 generates control information in accordance with the input signal. The control information generated by the remote control unit 201 includes information such as the angle of the tires as the driving unit of the mobile object 100, information about the accelerator opening and opening / closing timing, and information about the braking strength and braking timing.

[0058] Instead of the steering unit 302, a dedicated controller, a smartphone, a keyboard, a mouse, etc. may be used.

[0059] Furthermore, the determination of the moving direction of the moving object 100 and the remote control of the moving object 100 may be performed automatically or semi-automatically. For example, the moving direction may be automatically determined from within the movable area of ​​the moving object 100 based on the recognition result, and control information may be generated so that the moving object 100 moves in the determined moving direction. In this case, operator intervention is not required, and the remote control device 300 may not be provided. Furthermore, by incorporating this automatic remote control configuration into the moving object 100, the learning system 500 can be configured without providing the remote control unit 201 in the server device 200.

[0060] On the other hand, the movement direction of the moving body 100 can also be determined manually. For example, a manager or maintenance person of the moving body 100 may directly urge the moving body 100 by, for example, pushing the moving body 100 with their hands, thereby moving the moving body 100. Alternatively, a manager or maintenance person of the moving body 100 may remotely control the moving body 100 from near the moving body 100 using a radio control, an infrared control, or the like, to move the moving body 100. Furthermore, a manager or maintenance person of the moving body 100 may move the moving body 100 by operating a device such as a steering unit provided on the moving body 100 while on board the moving body 100.

[0061] 4 is a diagram showing an example of semi-automatic remote control according to an embodiment. As shown in FIG. 4, movement direction candidates (thick arrows and thick dashed lines in the figure) may be automatically generated and displayed from sensor information and recognition results, and the movement direction may be determined by having the operator select an appropriate one from these candidates. In this case, there is an advantage in that the configuration of the user interface can be easily simplified.

[0062] The control information transmitting unit 202 transmits the control information transmitted from the remote control unit 201 to the control information receiving unit 102 in the moving object 100. The control information receiving unit 102 transmits the control information transmitted from the control information transmitting unit 202 to the control unit 101. Then, the control unit 101 drives the driving units and the like in accordance with the control information transmitted from the control information receiving unit 102 to move the moving object 100.

[0063] Here, the control information generated as described above through the intervention of a person or the function of the relatively high-performance server device 200 is information that enables the mobile object 100 to move appropriately. Furthermore, when generating the control information, the sensor information transmitted to the remote control unit 201 includes information that the mobile object directly obtains from the surrounding environment in order to determine the direction of movement of the mobile object 100. That is, by using the sensor information as input data for training data and adding the control information as annotation information to the input data, it is possible to automatically generate training data to which annotation information has been added. That is, the generation of information for machine learning described below is performed using at least the sensor information and the control information.

[0064] The learning data generation unit 204 acquires the sensor information transmitted from the sensor information receiving unit 203 and the control information input from the remote control unit 201. The function of the learning data generation unit 204 to acquire information is an example of what realizes the function of an acquisition unit.

[0065] The training data generation unit 204 also acquires the recognition results transmitted from the sensor information receiving unit 203. Teacher data (fourth information) for machine learning is generated by adding, as annotation information to the sensor information, area information on the sensor information corresponding to the area where the mobile object 100 has actually moved (area where the mobile object 100 can move) and information indicating the difficulty of movement in the surrounding environment. The function of the training data generation unit 204 related to generating teacher data is, in other words, an example of realizing the function of the generation unit.

[0066] The learning data generation unit 204 further estimates, from the sensor information acquired by the sensing unit 104 when the moving object 100 moved based on the control information, whether the movement based on this control information was appropriate, i.e., whether it was actually possible to move through this area, and generates an estimation result (third information). The function of this learning data generation unit 204 related to generating the estimation result is, in other words, an example of realizing the function of an estimation unit. The learning data generation unit 204 is an example of an information generation device having the functions of an acquisition unit, a generation unit, and an estimation unit.

[0067] For example, the estimation is based on the difficulty of movement of the moving body 100, which is estimated based on the number of times the moving body 100 brakes and changes direction while moving in accordance with the control information, the magnitude and number of times vibrations and the like that have an adverse effect on the movement of the moving body, the number of other moving bodies, etc. For example, if the number of times the brakes were actually used is greater than a threshold set based on the moving distance of the moving body 100, such as 1, 2, 3, 5, or 10 times, an estimation result is generated indicating that movement in this area was impossible, and if the number is less than that, an estimation result is generated indicating that movement in this area was possible.

[0068] Similarly, for example, for the number of direction changes, the magnitude and number of detections of vibrations that adversely affect the movement of the moving body, and the number of other moving bodies, if the difference is within a predetermined range with respect to a threshold set based on the distance the moving body moved, an estimated result is generated indicating that movement in this area was possible, and if the difference exceeds the predetermined range, an estimated result is generated indicating that movement in this area was not possible.

[0069] The estimation result generated in this manner is used to verify whether the moving object 100 was actually able to move through this area. For example, if the control information indicates that the moving object 100 is unable to move, the control information is annotated with information indicating that the moving object is unable to move. If only annotation information indicating that the moving object is able to move is to be used in machine learning, the training data annotated with the annotation information regarding the control information should be excluded. Furthermore, if the learning model is a model for distinguishing (clustering) whether the area corresponds to an area in which the moving object is able to move or an area in which the moving object is unable to move, both annotation information indicating that the moving object is able to move and annotation information indicating that the moving object is unable to move can be used. In this manner, the estimation result generated above is used to determine the type of annotation information.

[0070] The information on the difficulty of movement (fifth information) described above may also be used. Specifically, even if a combination of sensor information, control information, and estimation results satisfies the threshold criteria and allows movement, if the movement of the moving body 100 is relatively difficult, other factors may make the movement impossible. In other words, a combination of sensor information, control information, and estimation results that results in a high degree of difficulty of movement is unreliable. Therefore, such a combination of sensor information, control information, and estimation results may not be used for machine learning. Alternatively, such a combination of sensor information, control information, and estimation results may be considered to have low reliability and used for machine learning with a different reliability from other combinations of sensor information, control information, and estimation results. Furthermore, the difficulty of movement may be used for clustering of movable areas. For example, movable areas that are movable but have a high degree of difficulty of movement may be classified as "areas requiring careful movement," and movable areas that are movable but have a low degree of difficulty of movement may be classified as "areas where movement is easy," and each of these may be used for machine learning.

[0071] On the other hand, if it is possible to accurately determine whether movement is possible in an area where the combination of sensor information, control information, and estimation results makes movement difficult, it is possible to expand the area in which the moving body 100 can move. In other words, in a situation where the benefits of expanding the area in which movement is possible are emphasized, such as when the area in which movement is possible is small or when there are few alternative movement areas, combinations of sensor information, control information, and estimation results that make movement difficult may be preferentially used in machine learning. In order to accurately determine whether movement is possible in an area where the combination of sensor information, control information, and estimation results makes movement difficult, it is necessary to generate a sufficient amount of training data necessary for learning, and the effect of the information generation method of this embodiment can be significant.

[0072] Note that being used preferentially for machine learning means that machine learning is performed on the teacher data of other combinations after all machine learning using the teacher data of combinations of sensor information, control information, and estimation results with high difficulty has been completed. In this case, machine learning on the teacher data of other combinations may be omitted. The teacher data of other combinations is teacher data for areas where movement is relatively easy. In other words, in such areas, autonomous movement based on rules based on sensor information may be sufficient to move around without relying on machine learning, so omitting machine learning on the teacher data of other combinations may be acceptable depending on the conditions.

[0073] Another method of prioritizing machine learning is to first use training data with a low level of difficulty and then use training data with a high level of difficulty. Depending on the machine learning conditions, the influence of data learned later in the time series may be greater than when the data is used preferentially in machine learning. This method is effective in such cases.

[0074] Furthermore, after learning using all the training data, further training may be performed using training data with higher difficulty. Furthermore, a bias may be applied so that training data with higher difficulty have a greater influence on the learning results. For example, the ratio of the number of training data with higher difficulty to the number of training data with lower difficulty may be adjusted so that the number of training data with higher difficulty is greater, and the training data with lower difficulty may be thinned out.

[0075] To summarize the above explanation, in this embodiment, when the combination of sensor information and control information satisfies predetermined conditions, the estimation result of whether the moving body 100 can move and the difficulty of the movement of the moving body 100 both satisfy predetermined conditions, and training data to which annotation information for machine learning is added is generated. The training data generation unit 204 transmits the training data generated in this manner to the storage device 206 and stores it.

[0076] FIG. 5 is a first diagram illustrating an example of training data according to an embodiment. FIG. 6 is a second diagram illustrating an example of training data according to an embodiment. As shown in FIG. 5, the training data generated as described above adds annotation information indicating that the area in which the moving object 100 was previously able to move is an area in which the moving object 100 is able to move (coarse dot hatching in the figure). As such, the training data output in this embodiment may include a composite image in which an image of the area in which the moving object 100 moves, included in the sensor information, is divided into multiple sections, and information indicating whether the moving object 100 is able to move based on the estimation result is superimposed on the image. For this reason, for example, information (sixth information) dividing the area in which the moving object 100 moves into multiple sections by type may be acquired. This information may be generated separately, for example, by using the recognition result or the like.

[0077] When determining the area in which the moving object 100 can move, an area with a predetermined width along the moving direction may be used by superimposing the history of when the same area has been passed multiple times, or an area may be used by combining the moving area in one pass with the width of the moving object 100 itself, as shown in Fig. 6. The correspondence between the area in which the moving object 100 has moved and the pixels on the image may be determined by combining control information, sensor information, and additional information such as an ID and a timestamp for matching the information.

[0078] Furthermore, an area where a moving body (another moving body) other than the moving body 100 moves in the surrounding environment may be used for adding annotation information. Specifically, a moving body that is sufficiently larger than the moving body 100 and moves at a sufficiently high speed may be identified as a dangerous moving body, and an area where such a dangerous moving body moves may be set as an area where movement is prohibited. An example of such a dangerous moving body is an automobile, in contrast to the moving body 100 for transporting luggage. Alternatively, an automobile may be directly identified using a technique such as pattern matching.

[0079] Alternatively, a moving body that is the same size as the moving body 100 and does not move at a very high speed may be identified as a trackable moving body, and the area in which such a trackable moving body moves may be determined as the movable area. Examples of such trackable moving bodies include bicycles and wheelchairs in comparison with the moving body 100 for transporting luggage. Bicycles and wheelchairs may also be directly identified using techniques such as pattern matching.

[0080] Furthermore, annotation information may be added not only to a certain part of the sensor information (such as a certain area in an image of one frame, i.e., some pixels), but also to the entire sensor information (such as the entire image of one frame). This makes it possible to build a learning model that, for example, inputs one image as sensor information and outputs a judgment result indicating whether movement is possible or impossible, or whether movement is easy or difficult.

[0081] The learning unit 205 uses the training data with annotation information added thereto and stored in the storage device 206 to train a learning model through machine learning. After this learning is completed, the learning unit 205 outputs the trained learning model to the recognition unit.

[0082] Next, the above-described learning system 500 will be described with reference to Fig. 7. Fig. 7 is a flowchart showing the processing of the learning system according to the embodiment.

[0083] 7, first, the learning data generating unit 204 acquires sensor information and control information (acquisition step S101). The sensor information and control information are acquired from the moving object via the sensor information acquiring unit 203.

[0084] Next, the learning data generation unit 204 acquires sensor information and recognition results of the moving body 100 when it moves in accordance with the control information, and based on this sensor information and recognition results, estimates whether the area in which the moving body 100 moves in accordance with the control information is movable (estimation step S102).

[0085] Next, the learning data generating unit 204 generates training data to which annotation information has been added based on the sensor information, the control information, and the estimation result in the estimation step S102 (generation step S103).

[0086] The teacher data generated in this way is annotated with appropriate annotation information, so it can be used as is as teacher data for machine learning. Furthermore, since the generation of this teacher data is performed automatically, personnel costs are reduced, and a large amount of teacher data can be generated at once using multiple moving bodies 100, etc.

[0087] While the information generation method according to one or more aspects has been described above based on the embodiments, the present disclosure is not limited to these embodiments. As long as it does not deviate from the spirit of the present disclosure, various modifications conceivable by a person skilled in the art to the present embodiments and forms constructed by combining components of different embodiments may also be included within the scope of one or more aspects.

[0088] For example, by providing a mobile body with all of the functions of the server device in the above embodiment, it is possible to efficiently generate information for a learning model that estimates whether a mobile body can move through a specified area on a standalone basis.

[0089] Furthermore, in the above-described embodiments, each component may be configured with dedicated hardware, or may be realized by executing a software program suitable for each component. Each component may be realized by a program execution unit such as a CPU or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory. Here, the software that realizes the information generation device of the above-described embodiments is the following program.

[0090] That is, this program is a program that causes a computer to execute an information generating method. [Industrial Applicability]

[0091] The present disclosure can be used to train a learning model that is used to determine a movement area of ​​a self-propelled mobile body. [Explanation of symbols]

[0092] 100 Mobile 101 Control section 102 Control information receiving unit 103 Sensor information transmission unit 104 Sensing unit 105 Recognition part 150 Network 200 Server device 201 Remote Control Unit 202 Control information transmission unit 203 Sensor information receiving unit 204 Learning Data Generation Unit 205 Learning Department 206 Storage Devices 300 Remote Control Device 301 Display device 302 Steering section 500 Learning System

Claims

1. 1. An information generation method in an information generation device for generating information for a learning model that estimates whether a moving object can move through a predetermined area, comprising: When a moving object moves through a first area, (1) first information acquired from at least a sensor installed on the moving object, and (2) second information related to the movement of the moving object are acquired; estimating whether the moving object is capable of moving through the first area according to the second information; generating fourth information for the learning model in which the first information and the second information are associated with third information indicating a result of estimation as to whether or not the movement is possible; Information generation method.

2. The estimation of whether the moving object is capable of moving through the first area is based on whether a difference between movement data during movement through the first area included in the first information obtained when the moving object moves according to the second information and a threshold value is within a predetermined range. The information generating method according to claim 1 .

3. The second information is input by an operator who remotely controls the mobile object. The information generating method according to claim 2 .

4. Furthermore, a degree of difficulty of movement of the moving object is estimated, In generating the fourth information, the fourth information is generated by associating the first information, the second information, fifth information indicating a result of estimation of difficulty, and the third information. The information generating method according to any one of claims 1 to 3.

5. The fourth information is generated only when the third information and the fifth information satisfy a predetermined condition. The information generating method according to claim 4.

6. The fourth information includes a reliability estimated based on the first information. The information generating method according to any one of claims 1 to 5.

7. Further, sixth information is acquired that divides the first area into a plurality of sections by type; The fourth information includes a composite image in which, for each of a plurality of sections obtained by dividing an image of the first area included in the first information by the sixth information, information indicating whether the moving object based on the third information is movable is superimposed on the image of the first area. The information generating method according to any one of claims 1 to 6.

8. Identifying other moving objects present in the first area based on the first information; In the estimation of whether or not the first area is movable, (a) if the other moving object satisfies a first condition, estimating that the other moving object is capable of moving through the first region in which the other moving object exists; and (b) if the other moving object satisfies a second condition, estimating that the other moving object is not able to move through the first area in which the other moving object exists. The information generating method according to any one of claims 1 to 7.

9. A method for causing a computer to execute the information generating method according to any one of claims 1 to 8. program.

10. An information generation device for generating information for a learning model that estimates whether a moving object can move through a predetermined area, comprising: an acquisition unit that acquires, when a moving object moves through a first area, (1) first information acquired from at least a sensor installed on the moving object, and (2) second information regarding the first area; an estimation unit that estimates whether the moving object is capable of moving through the first area based on the first information; a generation unit that generates fourth information for the learning model by associating the first information and the second information with third information that indicates a result of estimation of whether or not the object is movable. Information generation device.

Images