Machine learning methods

Abstract

To improve the techniques used in machine learning. [Solution] The information processing device 10 performs a first determination process to determine the presence or absence of an object in the captured image based on the time-series data of the captured image. Next, the information processing device 10 performs a second determination process using a first learning model to determine the presence or absence of the object in the captured image based on the time-series data. If the results regarding the presence or absence of the object differ between the first determination process and the second determination process, the information processing device 10 performs a third determination process using the second learning model to determine the presence or absence of the object in the captured image. Then, based on the result of the third determination process, the information processing device 10 adds the captured image to the first learning model.

Description

Technical Field

[0001] The present disclosure relates to a machine learning method executed by an information processing apparatus.

Background Art

[0002] Conventionally, techniques used in machine learning are known. For example, in Patent Document 1, learning is performed by an object recognition method using reference data including a specific identification target, and an identification model for creating an identification model of the specific identification target is used to detect the specific identification target from video data including the specific identification target by the object recognition method, and teacher data of the specific identification target is generated.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Improvements are desired in the techniques used in machine learning.

[0005] In view of such circumstances, an object of the present disclosure is to improve the techniques used in machine learning.

Means for Solving the Problems

[0006] A machine learning method according to one embodiment of the present disclosure is a machine learning method executed by an information processing device, comprising: a first determination process to determine the presence or absence of an object in an captured image based on time-series data of the captured image; a second determination process using a first learning model to determine the presence or absence of the object in the captured image based on the time-series data of the captured image; if the results regarding the presence or absence of the object differ between the first determination process and the second determination process, a third determination process to determine the presence or absence of the object in the captured image using the second learning model; and adding the captured image to the first learning model based on the result of the third determination process. [Effects of the Invention]

[0007] According to one embodiment of this disclosure, the techniques used in machine learning are improved. [Brief explanation of the drawing]

[0008] [Figure 1] This is a block diagram showing the schematic configuration of the information processing device related to this disclosure. [Figure 2] This is a flowchart showing the operation of the information processing device related to this disclosure. [Modes for carrying out the invention]

[0009] The embodiments of this disclosure will be described below.

[0010] (Summary of the embodiment) Referring to Figure 1, the configuration of the information processing device 10 according to this embodiment will be described. The information processing device 10 is, for example, a server device, a PC (Personal Computer), a smartphone or other computer, or a general-purpose or dedicated electronic device. The information processing device 10 is connected via a network to a server that provides any learning model, such as the first learning model or the second learning model used in this embodiment.

[0011] First, an overview of this embodiment will be described, and details will be described later. The information processing device 10 determines the presence or absence of an object in the captured image based on the time-series data of the captured image through a first determination process. Next, the information processing device 10 determines the presence or absence of the object in the captured image based on the time-series data through a second determination process using a first learning model. If the results regarding the presence or absence of the object differ between the first determination process and the second determination process, the information processing device 10 executes a third determination process that determines the presence or absence of the object in the captured image using the second learning model. Then, based on the result of the third determination process, the information processing device 10 adds the captured image to the first learning model.

[0012] Object recognition methods using anomaly detection or anomaly estimation require the prior collection of training data for recognition. However, since there is little training data on anomalies, it is difficult to detect anomalies using object recognition methods. Therefore, a challenge is how to generate training data on anomalies. In response to this, according to this embodiment, the information processing device 10 uses two different determination processes, a first determination process and a second determination process, to determine the presence or absence of an object in the captured image, and if the determination results are different, it executes a third determination process. Then, based on the result of the third determination process, the information processing device 10 adds the captured image to the first learning model used in the second determination process. Therefore, for example, if only the second determination process using the first learning model determines that there is no object in the captured image, the captured image can be added as training data on the anomaly of an object being present, thus improving the opportunity to generate training data on the presence or absence of an object used in the first learning model. Furthermore, this can also improve the accuracy of the first learning model's determination of the presence or absence of an object. Therefore, according to this embodiment, the techniques used in machine learning are improved in that there is an opportunity to generate training data regarding the presence or absence of objects, and the accuracy of the first learning model's determination of the presence or absence of objects can be improved.

[0013] Next, the configuration of the information processing device 10 will be described in detail.

[0014] (Configuration of information processing device) The information processing device 10 comprises a communication unit 11, an output unit 12, an input unit 13, a storage unit 14, and a control unit 15.

[0015] The communication unit 11 includes one or more communication interfaces connected to a network. These communication interfaces are compatible with, but are not limited to, mobile communication standards such as 4G (4th Generation) or 5G (5th Generation). The communication unit 11 receives information used in the operation of the information processing device 10 and transmits information obtained through the operation of the information processing device 10. In this embodiment, the communication unit 11 also enables the information processing device 10 to communicate via the network with, for example, a terminal device, server, etc., that provides time-series data of captured images used to determine the presence or absence of an object. Furthermore, if the information processing device 10 uses a learning model located on another server, the communication unit 11 enables the information processing device 10 to communicate with that server via the network.

[0016] The output unit 12 includes one or more output devices that output information. These output devices are, for example, a display that outputs information as video, or a speaker that outputs information as sound, but are not limited to these. Alternatively, the output unit 12 may include an interface for connecting an external output device.

[0017] The input unit 13 includes one or more input devices for detecting user input operations. These input devices are, but are not limited to, physical keys, capacitive keys, mice, touch panels, touchscreens integrated with the display of the output unit 12, or microphones. Alternatively, the input unit 13 may include an interface for connecting external input devices.

[0018] The storage unit 14 includes one or more memories. The memory may be, for example, a semiconductor memory, a magnetic memory, an optical memory, or the like, but is not limited thereto. Each memory included in the storage unit 14 may function as, for example, a main storage device, an auxiliary storage device, or a cache memory. The storage unit 14 stores any information used for the operation of the information processing apparatus 10. Also, for example, the storage unit 14 may store a system program, an application program, and embedded software, etc. For example, the information stored in the storage unit 14 may be updated with information acquired from a network via the communication unit 11, for example. Also, the storage unit 14 may store normal data (Normal Data) and anomaly data (Anomaly Data) used for the first learning model as learning data. Further, the storage unit 14 may store the first learning model and / or the second learning model.

[0019] The control unit 15 includes at least one processor, a programmable circuit such as at least one FPGA (field-programmable gate array), a dedicated circuit such as at least one ASIC (application specific integrated circuit), or any combination thereof. The processor is a general-purpose processor such as a CPU (central processing unit) or a GPU (graphics processing unit) or a dedicated processor specialized for specific processing. The control unit 15 executes processing related to the operation of the information processing apparatus 10 while controlling each part of the information processing apparatus 10.

[0020] (Operation flow of the information processing apparatus) Referring to FIG. 2, the operation of the information processing apparatus 10 according to the present embodiment will be described.

[0021] S100: The control unit 15 of the information processing apparatus 10 determines the presence or absence of an object in the captured image based on the time-series data of the captured image by the first determination process.

[0022] Specifically, for example, when a user inputs time-series data of captured images using the input unit 13, the control unit 15 estimates the depth of each captured image included in the time-series data by depth estimation. Next, the control unit 15 calculates the difference in depth of each captured image in time series and operates on each difference. Then, the control unit 15 determines the presence or absence of an object based on whether the magnitude of the difference is greater than or equal to a first threshold value. If it is greater than or equal to the first threshold value, the control unit 15 determines that an object is present in the captured image. If it is less than the first threshold value, the control unit 15 determines that no object is present in the captured image. The time-series data of the captured images may be directly received, for example, from a vehicle traveling on a road via the communication unit 11, may use the data stored in the storage unit 14, or may be received from another server. Thus, the time-series data of the captured images obtained by any method can be used. Also, the first threshold value is any value with a high possibility of an object being present in the captured image. The first threshold value may be changeable. Also, an arbitrary learning model may be used for the first determination process.

[0023] S101: The control unit 15 determines the presence or absence of the object in the captured image based on the time-series data by a second determination process using a first learning model.

[0024] Specifically, the control unit 15 inputs the time-series data of the captured image acquired in S101 into the first learning model, which is an anomaly estimation model. Next, the control unit 15 obtains the estimated depth results for each captured image included in the time-series data output from the first learning model. Next, the control unit 15 calculates the difference in depth for each captured image over time and calculates the difference for each image. Then, the control unit 15 determines the presence or absence of an object based on whether the magnitude of the difference is greater than or equal to the second threshold. If it is greater than or equal to the second threshold, the control unit 15 determines that there is an object in the captured image. If it is less than the second threshold, the control unit 15 determines that there is no object in the captured image. The second threshold is an arbitrary value that indicates a high probability of the presence of an object in the captured image. The second threshold may be changeable. Furthermore, the output from the first learning model is not limited to depth results, but may be any output that can determine the presence or absence of an object, such as information indicating the region on the captured image where an object is estimated to be present. The first learning model may be an anomaly detection model, or any other learning model.

[0025] S102: The control unit 15 determines whether the results regarding the presence or absence of the object in the first determination process and the second determination process match.

[0026] If the results match (S102-Yes), the control unit 15 terminates the process, determining that the first learning model's judgment is correct. If the results differ (S102-No), the process proceeds to S103.

[0027] S103: If the results regarding the presence or absence of the object differ between the first and second determination processes (S102-No), the control unit 15 executes a third determination process to determine the presence or absence of the object in the captured image using the second learning model.

[0028] Specifically, for example, if the second learning model is a large-scale language model (LLM), the control unit 15 takes the captured image as input to the second learning model and inputs a prompt to query whether or not it is an object. Note that the second learning model may be any learning model different from the first learning model.

[0029] S104: The control unit 15 adds the captured image to the first learning model based on the result of the third determination process.

[0030] For example, if it is determined that an object is present in S100, absent in S101, and then determined that an object is present again in S103, the control unit 15 may add the captured image to the first learning model as training data for information related to the presence of an object. In this case, the determination of the first learning model is incorrect, and it is highly likely that an object exists in the captured image. Also, if it is determined that there is no object in S100, present in S101, and then absent in S103, the control unit 15 may add the captured image to the first learning model as training data for information related to the absence of an object. In this case, the determination of the first learning model is incorrect, and it is highly likely that an object does not exist in the captured image. Otherwise, the control unit 15 may terminate the process. In this case, it is highly likely that the determination of the first learning model is correct.

[0031] While this disclosure has been described based on the drawings and embodiments, it should be noted that those skilled in the art may make various modifications and alterations based on this disclosure. Therefore, it should be noted that these modifications and alterations are within the scope of this disclosure. For example, the functions, etc., included in each component or step can be rearranged in a logically consistent manner, and multiple components or steps can be combined into one or divided into two.

[0032] For example, in the embodiment described above, it is also possible to have an embodiment in which the configuration and operation of the information processing device 10 are distributed among multiple computers that can communicate with each other. For example, the information processing device 10 may be composed of multiple computers.

[0033] Furthermore, for example, in the embodiment described above, if the control unit 15 detects multiple objects from the captured image in the first and second determination processes, it may crop each portion of the multiple objects detected from the captured image. If the results regarding the presence or absence of multiple objects differ between the first and second determination processes, the control unit 15 may execute a third determination process to determine the presence or absence of objects in each of the cropped portions from the captured image with the different results.

[0034] Specifically, in steps S101 and S102, the control unit 15 divides the captured image into multiple regions and crops the regions in each region that are equal to or greater than the first threshold or the second threshold. Next, the control unit 15 determines whether all the cropped regions in steps S101 and S102 match. If the cropped regions do not match, the control unit 15 determines that one of the results regarding the presence or absence of multiple objects differs between the first determination process and the second determination process. In this case, the control unit 15 executes step S103 for each of the mismatched cropped regions. The control unit 15 may then add the captured image to the first learning model based on the combined results of step S103 for each of the mismatched cropped regions. For example, the control unit 15 may add the captured image to the first learning model only if the determinations regarding the presence or absence of objects in steps S101 and S103 for each of the mismatched cropped regions all match. Alternatively, the control unit 15 may add each cropped region to the first learning model based on the result of S103 in each of the cropped regions that did not match.

[0035] Furthermore, it is also possible to implement an embodiment in which a general-purpose computer functions as the information processing device 10 according to the above embodiment. Specifically, a program describing the processing content that realizes each function of the information processing device 10 according to the above embodiment is stored in the memory of the general-purpose computer, and the processor reads and executes the program. Therefore, this disclosure can also be implemented as a program that can be executed by a processor, or as a non-temporary computer-readable medium that stores the program. [Explanation of Symbols]

[0036] 10 Information processing unit, 11 Communication unit, 12 Output unit, 13 Input unit, 14 Storage unit, 15 Control unit

Claims

1. A machine learning method executed by an information processing device, The first determination process determines the presence or absence of an object in the captured image based on the time-series data of the captured image. A second determination process using a first learning model determines the presence or absence of the object in the captured image based on the time-series data. If the results regarding the presence or absence of the object differ between the first determination process and the second determination process, a third determination process is performed to determine the presence or absence of the object in the captured image using the second learning model, and Based on the result of the third determination process, the captured image is added to the first learning model. Machine learning methods, including those mentioned above.

2. A machine learning method according to claim 1, The information processing device is a machine learning method in which, if it is determined in the first determination process that an object is present, in the second determination process that an object is absent, and in the third determination process that an object is present, the captured image is added to the first learning model as training data for information related to the presence of the object.

3. A machine learning method according to claim 1, The information processing device is a machine learning method that, when it is determined in the first determination process that there is no object, in the second determination process that there is an object, and in the third determination process that there is no object, adds the captured image as training data for information related to the absence of the object to the first learning model.

4. A machine learning method according to claim 1, In the first and second determination processes, if multiple objects are detected from the captured image, crop each portion of the multiple objects detected from the captured image. It further includes, The information processing device is a machine learning method that, if any of the results regarding the presence or absence of the plurality of objects differ between the first determination process and the second determination process, executes a third determination process to determine the presence or absence of objects in each of the cropped portions from the captured image where the result differs.

5. A machine learning method according to claim 1, The first decision process is a machine learning method that uses depth estimation.

Images