Information processing device, generation method, information processing method, and program

Abstract

Provided is an information processing device (10) comprising: an identification means (11) that identifies, from a period of a video that is captured by an image capture device installed in a vehicle, a period in which information recognized from the video matches a rule corresponding to a specific scene; an acquisition means (12) that acquires a data set that is a combination of information recognized from the video of the period identified by the identification means and a correct answer label indicating whether or not the video of said period is of said specific scene; and a generation means (13) that performs training on the basis of the data set acquired by the acquisition means, and generates a trained model for estimating whether or not a video of a specific period is of the specific scene.

Description

[Technical Field]

[0001] The present disclosure relates to an information processing device, a generation method, an information processing method, and a non-transitory computer-readable medium having a program stored thereon. [Background technology]

[0002] Patent Document 1 discloses a technique for early detection of a moving object that may cut into the lane in which the vehicle is traveling, based on an image of the surroundings of the vehicle. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2020-003971 Summary of the Invention [Problem to be solved by the invention]

[0004] However, Patent Document 1 does not consider, for example, extracting moving images of scenes such as interruptions from accumulated (recorded, filmed) moving images.

[0005] In view of the above-mentioned problems, an object of the present disclosure is to provide a technology that can appropriately extract a video of a specific scene from stored videos. [Means for solving the problem]

[0006] In a first aspect of the present disclosure, an information processing device is provided that includes: an identification means for identifying a period of a video captured by a camera mounted on a vehicle in which information recognized from the video matches a rule corresponding to a specific scene; an acquisition means for acquiring a dataset that combines information recognized from the video for the period identified by the identification means and a correct label indicating whether the video for the period is a video of the specific scene; and a generation means for performing learning based on the dataset acquired by the acquisition means and generating a trained model that estimates whether the video for the specific period is a video of the specific scene.

[0007] In addition, a second aspect of the present disclosure provides an information processing device having an identification means for identifying a period of a video captured by a camera mounted on a vehicle in which information recognized from the video matches a rule corresponding to a specific scene, and an estimation means for estimating whether the video is a video of the specific scene based on the information recognized from the video of the period identified by the identification means and a trained model.

[0008] In addition, a third aspect of the present disclosure provides a generation method that identifies a period of a video captured by a camera mounted on a vehicle during which information recognized from the video matches a rule corresponding to a specific scene, obtains a dataset combining information recognized from the video for the identified period with a correct label indicating whether the video for that period is a video of the specific scene, performs learning based on the obtained dataset, and generates a trained model that estimates whether the video for the specific period is a video of the specific scene.

[0009] In addition, a fourth aspect of the present disclosure provides an information processing method that identifies a period of a video captured by a camera mounted on a vehicle during which information recognized from the video matches a rule corresponding to a specific scene, and estimates whether the video is a video of the specific scene based on the information recognized from the video during the identified period and a trained model.

[0010] In addition, a fifth aspect of the present disclosure provides a non-transitory computer-readable medium storing a program that causes a computer to execute the following process: identify a period of video captured by a camera mounted on a vehicle during which information recognized from the video matches a rule corresponding to a specific scene; obtain a dataset that combines information recognized from the video for the identified period with a correct label indicating whether the video for that period is a video of the specific scene; perform learning based on the obtained dataset; and generate a trained model that estimates whether the video for the specific period is a video of the specific scene.

[0011] In addition, a sixth aspect of the present disclosure provides a non-transitory computer-readable medium storing a program that causes a computer to execute a process of identifying a period of a video captured by a camera mounted on a vehicle during which information recognized from the video matches a rule corresponding to a specific scene, and estimating whether the video is a video of the specific scene based on the information recognized from the video during the identified period and a trained model. [Effects of the Invention]

[0012] According to one aspect, a video of a particular scene can be appropriately extracted from stored videos. [Brief explanation of the drawings]

[0013] [Figure 1] FIG. 1 is a diagram illustrating an example of a configuration of an information processing device that performs a learning process according to an embodiment. [Figure 2] FIG. 1 is a diagram illustrating an example of a configuration of an information processing device that performs estimation processing according to an embodiment. [Figure 3] FIG. 1 is a diagram illustrating an example of a hardware configuration of an information processing apparatus according to an embodiment. [Figure 4] FIG. 1 is a diagram illustrating an example of a configuration of an information processing system according to an embodiment. [Figure 5] 10 is a flowchart illustrating an example of a learning process of the information processing device according to the embodiment. [Figure 6]FIG. 2 is a diagram illustrating an example of a learning DB according to the embodiment. [Figure 7] 10 is a flowchart illustrating an example of an estimation process of the information processing device according to the embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0014] The principles of the present disclosure will be described with reference to some exemplary embodiments. It should be understood that these embodiments are set forth for illustrative purposes only, to aid those skilled in the art in understanding and practicing the present disclosure, without implying any limitation on the scope of the disclosure. The disclosure described herein may be implemented in various ways other than those described below. In the following description and claims, unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. Hereinafter, embodiments of the present disclosure will be described with reference to the drawings.

[0015] (Embodiment 1) <Configuration> <<Configuration of information processing device 10 that performs learning processing>> The configuration of an information processing device 10 that performs learning processing according to the embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of the configuration of the information processing device 10 that performs learning processing according to the embodiment. The information processing device 10 has an identification unit 11, an acquisition unit 12, and a generation unit 13. Each of these units may be realized by cooperation between one or more programs installed in the information processing device 10 and hardware such as a processor 101 and a memory 102 of the information processing device 10.

[0016] The identification unit 11 identifies a period of video captured by a camera mounted on a vehicle during which information recognized from the video matches a rule corresponding to a specific scene. The acquisition unit 12 acquires a dataset of combinations of information recognized from the video during the period identified by the identification unit 11 and a correct answer label indicating whether the video during that period is a video of the specific scene. The generation unit 13 performs learning based on the dataset acquired by the acquisition unit 12, and generates a trained model that estimates whether a video during the specific period is a video of the specific scene.

[0017] <<Configuration of information processing device 20 that performs estimation processing>> Next, the configuration of the information processing device 20 that performs the estimation processing according to the embodiment will be described with reference to Fig. 2. Fig. 2 is a diagram showing an example of the configuration of the information processing device 20 that performs the estimation processing according to the embodiment. The information processing device 20 has an identification unit 21 and an estimation unit 22. These units may be realized by cooperation between one or more programs installed in the information processing device 20 and hardware such as a processor and memory of the information processing device 20.

[0018] The identification unit 21 identifies a period of time during which information recognized from the video matches a rule corresponding to a specific scene, from among the periods of the video captured by the imaging device 31 mounted on the vehicle 30. The estimation unit 22 estimates whether the video is a video of the specific scene, based on the information recognized from the video of the period identified by the identification unit 21 and the trained model.

[0019] <Hardware configuration> 3 is a diagram showing an example of the hardware configuration of the information processing device 10 and the information processing device 20 according to the embodiment. The following description will be given using the information processing device 10 as an example. Note that the hardware configuration of the information processing device 20 may be the same as the hardware configuration of the information processing device 10.

[0020] 3, the information processing device 10 (computer 100) includes a processor 101, a memory 102, and a communication interface 103. These components may be connected via a bus or the like. The memory 102 stores at least a portion of a program 104. The communication interface 103 includes an interface required for communication with other network elements.

[0021] When the program 104 is executed by the processor 101, memory 102, and other components in cooperation with each other, the computer 100 performs at least some of the processing of the embodiments of the present disclosure. The memory 102 may be of any type suitable for a local technology network. The memory 102 may be, by way of non-limiting example, a non-transitory computer-readable storage medium. The memory 102 may also be implemented using any suitable data storage technology, such as semiconductor-based memory devices, magnetic memory devices and systems, optical memory devices and systems, fixed memory, and removable memory. While only one memory 102 is shown in the computer 100, several physically distinct memory modules may be present in the computer 100. The processor 101 may be of any type. The processor 101 may include one or more of a general-purpose computer, a special-purpose computer, a microprocessor, a digital signal processor (DSP), and, by way of non-limiting example, a processor based on a multi-core processor architecture. The computer 100 may have multiple processors, such as application-specific integrated circuit chips time-slaved to a clock that synchronizes the main processor.

[0022] Embodiments of the present disclosure may be implemented in hardware or special purpose circuits, software, logic, or any combination thereof. Some aspects may be implemented in hardware, while other aspects may be implemented in firmware or software that may be executed by a controller, microprocessor, or other computing device.

[0023] The present disclosure also provides at least one computer program product tangibly stored on a non-transitory computer-readable storage medium. The computer program product includes computer-executable instructions, such as instructions included in program modules, that execute on a target real or virtual processor or device to perform the processes or methods of the present disclosure. Program modules include routines, programs, libraries, objects, classes, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The functionality of the program modules may be combined or divided among program modules as desired in various embodiments. The machine-executable instructions of the program modules may be executed in local or distributed devices. In a distributed device, the program modules may be located in both local and remote storage media.

[0024] The program code for executing the methods of the present disclosure may be written in any combination of one or more programming languages. The program code may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus. When the program code is executed by the processor or controller, the functions / acts in the flowcharts and / or implementing block diagrams are performed. The program code may be executed entirely on the machine, partly on the machine, as a standalone software package, partly on the machine and partly on a remote machine, or entirely on a remote machine or server.

[0025] The program can be stored and provided to a computer using various types of non-transitory computer-readable media. Non-transitory computer-readable media include various types of tangible recording media. Examples of non-transitory computer-readable media include magnetic recording media, magneto-optical recording media, optical disk media, and semiconductor memory. Magnetic recording media include, for example, flexible disks, magnetic tapes, and hard disk drives. Magneto-optical recording media include, for example, magneto-optical disks. Optical disk media include, for example, Blu-ray discs, CD (Compact Disc)-ROMs (Read Only Memory), CD-Rs (Recordable), and CD-RWs (Rewritable). Semiconductor memory includes, for example, solid-state drives, mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, and RAMs (Random Access Memory). The program may also be provided to a computer by various types of temporary computer-readable media. Examples of temporary computer-readable media include electrical signals, optical signals, and electromagnetic waves. The temporary computer-readable medium can supply the program to the computer via a wired communication path such as an electric wire or an optical fiber, or via a wireless communication path.

[0026] (Embodiment 2) Next, the configuration of the information processing system 1 according to the embodiment will be described with reference to FIG. <System configuration> Fig. 4 is a diagram showing an example of the configuration of an information processing system 1 according to an embodiment. In the example of Fig. 4, the information processing system 1 includes an information processing device 10, an information processing device 20, and a vehicle 30. Note that the numbers of the information processing devices 10, the information processing devices 20, and the vehicles 30 are not limited to those shown in the example of Fig. 4.

[0027] 4, the information processing device 10, the information processing device 20, and the vehicle 30 are connected to each other so as to be able to communicate with each other via a network N. Examples of the network N include, for example, the Internet, a mobile communication system, a wireless LAN (Local Area Network), a short-range wireless communication such as BLE, a LAN, and a bus. Examples of the mobile communication system may include, for example, a fifth-generation mobile communication system (5G), a fourth-generation mobile communication system (4G), a third-generation mobile communication system (3G), a sixth-generation mobile communication system (6G), and the like.

[0028] The vehicle 30 is a vehicle that travels on a road. Examples of the vehicle 30 include, but are not limited to, an automobile, a motorcycle, a moped, and a bicycle. The vehicle 30 includes an image capturing device 31 and an ECU (Electronic Control Unit) 32.

[0029] The image capturing device 31 may be, for example, an in-vehicle camera that captures an image ahead of the vehicle 30. The image capturing device 31 may be provided, for example, on the dashboard of the vehicle 30. The image capturing device 31 may also be provided, for example, behind the rearview mirror of the vehicle 30.

[0030] The ECU 32 may, for example, perform control related to the driving of the vehicle 30. The ECU 32 may, for example, control the accelerator, brake, etc. based on the operation by the driver of the vehicle 30. Furthermore, the ECU 32 may, for example, automatically control the accelerator, brake, etc. of the vehicle 30. In this case, the ECU 32 may, for example, have the function of an Advanced Driver-Assistance Systems (ADAS). Furthermore, the ECU 32 may, for example, have the function of a specific level of autonomous driving.

[0031] For example, the ECU 32 may distribute video captured by the image capturing device 31 to the information processing device 10 or the like via wireless communication. In this case, the video captured by the image capturing device 31 may be acquired by the information processing device 10 via an external device (for example, a file server). Furthermore, the ECU 32 may record the video captured by the image capturing device 31 in a recording device. In this case, the recorded video may be recorded in the information processing device 10 or the like via a portable recording medium (for example, a Universal Serial Bus (USB) memory).

[0032] The information processing device 10 may be, for example, a device such as a server, a cloud, or a personal computer. The information processing device 10 recognizes subject information based on a video captured by a camera device 31 mounted on a vehicle 30. Then, the information processing device 10 identifies a period of a specific scene based on the recognized subject information and a preset rule. Then, the information processing device 10 performs machine learning based on a human judgment result as to whether the identified period is a specific scene.

[0033] The information processing device 20 may be, for example, a personal computer, a smartphone, a cloud, or other device. The information processing device 20 recognizes subject information based on a video captured by a camera device 31 mounted on a vehicle 30. The information processing device 20 then identifies a period of a specific scene based on the recognized subject information and a preset rule. The information processing device 20 then estimates whether the period is a specific scene based on the trained model generated by the information processing device 10 and the subject information for the identified period.

[0034] <Processing> <<Learning process>> Next, an example of a learning process of the information processing device 10 according to the embodiment will be described with reference to Fig. 5 and Fig. 6. Fig. 5 is a flowchart showing an example of a learning process of the information processing device 10 according to the embodiment. Fig. 6 is a diagram showing an example of a learning DB (database) 601 according to the embodiment.

[0035] In step S101, the identification unit 11 of the information processing device 10 identifies (recognizes) subject information, which is various information related to the subject, based on a video captured by the imaging device 31 mounted on the vehicle 30. Here, the identification unit 11 may, for example, perform image recognition on each frame of the video to image-recognize the type of subject, such as another vehicle, and its three-dimensional position relative to the vehicle 30. Then, the identification unit 11 may, for example, identify (calculate) the subject information based on the position of the subject on pixel coordinates in each frame of the video. In this case, the identification unit 11 may, for example, identify at least one of the distance between the vehicle 30 and the subject, the speed of the subject relative to the vehicle 30, and the position of the subject relative to the vehicle 30 as the subject information. Note that the identification unit 11 may, for example, identify the speed of the vehicle 30 based on information acquired from the ECU 32.

[0036] The identification unit 11 may identify information about the subject based on a video captured by the imaging device 31, which is, for example, a stereo camera. The identification unit 11 may also identify information about the subject using AI (Artificial Intelligence) based on a video captured by the imaging device 31, which is, for example, a single camera. The identification unit 11 may also identify information about the subject using information measured by LiDAR (Light Detection and Ranging) in addition to the video captured by the imaging device 31, for example.

[0037] Next, the identification unit 11 of the information processing device 10 identifies a period of the video captured by the imaging device 31 mounted on the vehicle 30 during which the subject information recognized from the video matches a rule (condition) corresponding to a specific scene (step S102). Here, the specific scene may include, for example, a scene in which another vehicle changes lane and the position of the other vehicle changes (cuts in, cuts in) in front of the vehicle 30. The specific scene may also include, for example, a scene in which another vehicle changes lane and the position of the other vehicle changes (cuts out, pulls out) from in front of the vehicle 30. The specific scene may also include, for example, a scene in which the vehicle 30 changes lanes.

[0038] Furthermore, rules according to specific scenes may be determined, for example, by an operator (administrator) and set in advance in the information processing device 10. In this case, for example, rules regarding cutting in may include the following (1) to (3): (1) The distance between the vehicle 30 and the other vehicle is 10 m or more. (2) The speed (relative speed) of the other vehicle relative to the vehicle 30 is 10 km / h or more. (3) The position of the other vehicle moves from the lane adjacent to the lane in which the vehicle 30 is traveling (ego lane) to the ego lane within 2 seconds.

[0039] The identification unit 11 may identify one or more periods of a second time length (e.g., several seconds) shorter than the first time length during which the video was captured by the imaging device 31, during which the subject information extracted from the video matches a rule corresponding to the specific scene, out of the first time length (e.g., 10 hours). This makes it possible to identify, for example, a video of a specific scene in the vehicle 30 that has been traveling for a predetermined period of time.

[0040] Next, the identification unit 11 of the information processing device 10 records data (records) of combinations of subject information recognized for the video of the identified period and a correct label indicating whether the video is a video of a specific scene in the learning DB 601 (step S103). Note that the learning DB 601 may be recorded in a storage device inside the information processing device 10 or in a storage device external to the information processing device 10.

[0041] In the example of FIG. 6, the learning DB 601 stores one or more pieces of subject information and a correct answer label in association with a combination of a video ID, a specific scene, and a period. The video ID is identification information of a video captured by the imaging device 31. The period is a period identified by the identification unit 11. The period may be represented by a set of data including a time length from the beginning of the video associated with the video ID to the start of the period and a time length from the beginning of the video associated with the video ID to the end of the period. The subject information may include, for example, the type of subject, the distance between the vehicle 30 and the subject, the speed of the subject relative to the vehicle 30, and the position of the subject relative to the vehicle 30, for each subject ID recognized in the video of the period. The subject ID is identification information of the subject. The subject ID may be assigned by the identification unit 11 when the subject is first recognized. The type of subject may include, for example, a vehicle, a motorcycle, a pedestrian, etc. The correct label is information indicating whether or not a video of a certain period is a video of a specific scene. The correct label may be set, for example, by a human visually checking the video of the period identified by the identification unit 11. This makes it possible to perform supervised learning based on the results of a human checking whether or not the video of each period identified by the identification unit 11 is a video of a specific scene.

[0042] Next, the acquisition unit 12 of the information processing device 10 acquires learning data from the learning DB 601 (step S104). Here, the acquisition unit 12 acquires a data set of combinations of information recognized for the video extracted by the identification unit 11 and a correct answer label indicating whether the video is a video of a specific scene.

[0043] Next, the generation unit 13 of the information processing device 10 performs learning based on the data set acquired by the acquisition unit 12, thereby generating a trained model that estimates whether or not a video is a specific scene (step S105). As a result, for example, when the accuracy of the rule-based determination by the identification unit 11 that a video is a specific scene is relatively low, it is possible to learn, through supervised learning, whether each video extracted by the identification unit 11 is truly a video of a specific scene. The generation unit 13 may perform learning using, for example, principal component analysis and clustering. Furthermore, the generation unit 13 may perform learning using, for example, a neural network (NN).

[0044] <<Inference (inference) processing>> Next, an example of the estimation process of the information processing device 20 according to the embodiment will be described with reference to Fig. 7. Fig. 7 is a flowchart showing an example of the estimation process of the information processing device 20 according to the embodiment.

[0045] In step S201, the identification unit 21 of the information processing device 20 identifies (recognizes) subject information, which is various information related to a subject, based on a video captured by the imaging device 31 mounted on the vehicle 30. The processing of step S201 may be the same as the processing of step S101 in FIG.

[0046] Next, the identification unit 21 of the information processing device 20 identifies a period of the video captured by the imaging device 31 mounted on the vehicle 30 in which information recognized from the video matches a rule corresponding to a specific scene (step S202). The process of step S202 may be the same as the process of step S102 in FIG. 5.

[0047] Next, the estimation unit 22 of the information processing device 20 estimates whether or not a video of a specific period is a video of a specific scene based on a trained model that estimates whether or not the video is a video of a specific scene and the subject information recognized from the video of the period identified by the identification unit 21 (step S203). Note that the trained model is a trained model generated by the information processing device 10 in the processing of Fig. 5. The subject information may include, for example, the distance between the vehicle 30 and the subject, the speed of the subject relative to the vehicle 30, and the position of the subject relative to the vehicle 30, which are recognized from the video of the period.

[0048] This allows, for example, a video of a specific scene to be appropriately extracted from videos captured by the imaging device 31 and stored. This allows, for example, engineers (developers, users) to easily check whether the advanced driver assistance system or autonomous driving function of the vehicle 30 is operating appropriately when actually traveling on a road. Also, for example, it allows insurance companies and the like to easily check whether the driver of the vehicle 30 is driving appropriately when actually traveling on a road.

[0049] For example, there may be cases where sufficient accuracy cannot be obtained by identifying only based on rules defined (specified, designed) by humans, etc. On the other hand, according to the present disclosure, learning is performed using the rule-based identification results and correct labels assigned by humans, etc., and the rule-based identification results are input to a trained model to estimate (infer) whether the identification results are correct. Therefore, for example, the accuracy of identifying a specific scene can be improved compared to when only identification is based on rules.

[0050] Furthermore, if identification is performed only using a trained model without rule-based identification, the load of the learning process and estimation process may be large. Furthermore, it may be necessary to prepare a huge amount of training data during learning. On the other hand, according to the present disclosure, rule-based identification is performed in the pre-machine learning stage (pre-processing) of the learning process and estimation process. Therefore, for example, the amount of data of a video that is the target of estimation of whether or not it is a specific scene using a trained model can be reduced, thereby reducing the load of the learning process and estimation process. Furthermore, the amount of training data that needs to be prepared during learning can be reduced.

[0051] <Modification> The information processing device 10 and the information processing device 20 may each be a device included in a single housing, but the information processing device 10 and the information processing device 20 of the present disclosure are not limited to this. Each unit of the information processing device 10 and the information processing device 20 may be realized, for example, by cloud computing configured with one or more computers. In this case, at least a part of the processing of each unit of the information processing device 10 and the information processing device 20 may be executed by an external device connected via a network. In this case, for example, the processing of identifying (recognizing) subject information based on a moving image in step S101 of FIG. 5 and step S201 of FIG. 7 may be executed using an API (Application Programming Interface) provided by the external device. Furthermore, for example, the information processing device 10 and the information processing device 20 may be the same device. Each of these information processing devices 10 and the information processing device 20 is also included in an example of the "information processing device" of the present disclosure.

[0052] The present disclosure is not limited to the above-described embodiment, and can be modified as appropriate within the scope of the present disclosure.

[0053] A part or all of the above-described embodiments can be described as, but not limited to, the following supplementary notes. (Appendix 1) An identification means for identifying a period of a video captured by a camera mounted on a vehicle in which information recognized from the video matches a rule corresponding to a specific scene; an acquisition means for acquiring a data set of combinations of information recognized from videos of the period identified by the identification means and correct labels indicating whether the videos of the period are videos of the specific scene; A generation means for performing learning based on the dataset acquired by the acquisition means and generating a trained model that estimates whether a video of a specific period is a video of the specific scene; An information processing device having the above. (Appendix 2) An identification means for identifying a period of a video captured by a camera mounted on a vehicle in which information recognized from the video matches a rule corresponding to a specific scene; an estimation means for estimating whether a video is a video of the specific scene based on information recognized from the video of the period identified by the identification means and a trained model; An information processing device having the above. (Appendix 3) The trained model is a trained model generated by learning based on a dataset of a combination of information recognized from a video of a certain period and a correct answer label indicating whether the video of the period is a video of the specific scene. 3. The information processing device according to claim 2. (Appendix 4) The information recognized from the video captured by the imaging device includes information indicating at least one of the distance between the vehicle and the subject, the speed of the subject relative to the vehicle, and the position of the subject relative to the vehicle. 4. The information processing device according to claim 1. (Appendix 5) the rule according to the specific scene includes information indicating at least one of a distance between the vehicle and a subject, a speed of the subject relative to the vehicle, and a change in the position of the subject relative to the vehicle; 5. The information processing device according to any one of claims 1 to 4. (Appendix 6) the specifying means specifies, from a first time period during which the video was shot by the imaging device, a second time period shorter than the first time period during which information recognized from the video matches a rule corresponding to the specific scene; 6. An information processing device according to any one of appendices 1 to 5. (Appendix 7) The specific scene includes at least one of a scene in which the position of another vehicle changes to a position in front of the vehicle due to a lane change of the other vehicle, a scene in which the position of the other vehicle changes from a position in front of the vehicle due to a lane change of the other vehicle, and a scene caused by a lane change of the vehicle. 7. An information processing device according to any one of claims 1 to 6. (Appendix 8) Identifying a period of a video captured by a camera mounted on the vehicle in which information recognized from the video matches a rule corresponding to a specific scene; Obtain a dataset of combinations of information recognized from videos of a specified period and correct labels indicating whether the videos of the specified period are videos of the specified scene; Performing learning based on the acquired dataset, and generating a trained model that predicts whether a video of a specific period is a video of the specific scene. Generation method. (Appendix 9) Identifying a period of a video captured by a camera mounted on the vehicle in which information recognized from the video matches a rule corresponding to a specific scene; Based on information recognized from a video of a specified period and a trained model, it is estimated whether the video is a video of the specified scene. Information processing methods. (Appendix 10) Identifying a period of a video captured by a camera mounted on the vehicle in which information recognized from the video matches a rule corresponding to a specific scene; Obtain a dataset of combinations of information recognized from videos of a specified period and correct labels indicating whether the videos of the specified period are videos of the specified scene; Performing learning based on the acquired dataset, and generating a trained model that predicts whether a video of a specific period is a video of the specific scene. A non-transitory computer-readable medium that stores a program that causes a computer to execute a process. (Appendix 11) Identifying a period of a video captured by a camera mounted on the vehicle in which information recognized from the video matches a rule corresponding to a specific scene; Based on information recognized from a video of a specified period and a trained model, it is estimated whether the video is a video of the specified scene. A non-transitory computer-readable medium that stores a program that causes a computer to execute a process. [Explanation of symbols]

[0054] 1. Information Processing Systems 10. Information processing equipment 11 Specific section 12 Acquisition Department 13 Generation part 20 Information processing equipment 21 Specific section 22 Estimation part 30 vehicles 31 Imaging equipment 32 ECU

Claims

1. An identification means for identifying a period of a video captured by a camera mounted on a vehicle in which information recognized from the video matches a rule corresponding to a specific scene; an acquisition means for acquiring a data set of combinations of information recognized from videos of the period identified by the identification means and correct labels indicating whether the videos of the period are videos of the specific scene; A generation means for performing learning based on the dataset acquired by the acquisition means and generating a trained model that estimates whether a video of a specific period is a video of the specific scene; and The specific scene includes at least one of a scene in which the position of another vehicle changes from in front of the vehicle due to a lane change of the other vehicle, and a scene in which the vehicle changes lanes. Information processing device.

2. An identification means for identifying a period of a video captured by a camera mounted on a vehicle in which information recognized from the video matches a rule corresponding to a specific scene; an estimation means for estimating whether a video is a video of the specific scene based on information recognized from the video of the period identified by the identification means and a trained model; An information processing device having the above.

3. The trained model is a trained model generated by learning based on a dataset of a combination of information recognized from a video of a certain period and a correct answer label indicating whether the video of the period is a video of the specific scene. The information processing device according to claim 2 .

4. The information recognized from the video captured by the imaging device includes information indicating at least one of the distance between the vehicle and the subject, the speed of the subject relative to the vehicle, and the position of the subject relative to the vehicle. The information processing device according to claim 1 .

5. the rule according to the specific scene includes information indicating at least one of a distance between the vehicle and a subject, a speed of the subject relative to the vehicle, and a change in the position of the subject relative to the vehicle; The information processing device according to claim 1 .

6. the specifying means specifies, from a first time period during which the video was shot by the imaging device, a second time period shorter than the first time period during which information recognized from the video matches a rule corresponding to the specific scene; The information processing device according to claim 1 .

7. The specific scene includes a scene in which another vehicle changes its position to be in front of the vehicle due to a lane change of the other vehicle. The information processing device according to claim 1 .

8. Identifying a period of a video captured by a camera mounted on the vehicle in which information recognized from the video matches a rule corresponding to a specific scene; Obtain a dataset of combinations of information recognized from videos of a specified period and correct labels indicating whether the videos of the specified period are videos of the specified scene; Learning is performed based on the acquired dataset, and videos from a specific period are identified as videos of the specific scene. The specific scene includes at least one of a scene in which the position of another vehicle changes from in front of the vehicle due to a lane change of the other vehicle, and a scene in which the vehicle changes lanes. Generation method.

9. Identifying a period of a video captured by a camera mounted on the vehicle in which information recognized from the video matches a rule corresponding to a specific scene; Based on information recognized from a video of a specified period and a trained model, estimate whether the video is a video of the specified scene; The specific scene includes at least one of a scene in which the position of another vehicle changes from in front of the vehicle due to a lane change of the other vehicle, and a scene in which the vehicle changes lanes. Information processing methods.

10. Identifying a period of a video captured by a camera mounted on the vehicle in which information recognized from the video matches a rule corresponding to a specific scene; Obtain a dataset of combinations of information recognized from videos of a specified period and correct labels indicating whether the videos of the specified period are videos of the specified scene; Performing learning based on the acquired dataset, and generating a trained model that predicts whether a video of a specific period is a video of the specific scene. Have the computer execute the process, The specific scene includes at least one of a scene in which the position of another vehicle changes from in front of the vehicle due to a lane change of the other vehicle, and a scene in which the vehicle changes lanes. program.

Images