Information processing device and information processing method

By estimating events from real-space images and constructing virtual space images with annotated events, the information processing device addresses the inefficiencies and accuracy issues of real-space imaging, creating highly accurate training data for image recognition models.

JP7954320B2Active Publication Date: 2026-11-19NTT DOCOMO INC
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
JP2025575167
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2024-02-07
Publication Date
2026-11-19
Estimated Expiration
2044-02-07

AI Technical Summary

Technical Problem

Obtaining images for training data by imaging in the real space is time-consuming and costly, and annotations from real-space images have low accuracy, while annotations from virtual space images are highly accurate but struggle to comprehensively reproduce real-world situations.

Method used

An information processing device that estimates events from real-space images using a first image recognition model, configures a virtual recognition target in a virtual space, acquires virtual space images, and constructs training data by annotating events onto these images to create highly accurate training data for image recognition models.

Benefits of technology

This approach allows for the creation of highly accurate training data with higher resolution than real-space images, improving the accuracy of machine learning for image recognition models.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007954320000001
    Figure 0007954320000001
  • Figure 0007954320000002
    Figure 0007954320000002
  • Figure 0007954320000003
    Figure 0007954320000003
Patent Text Reader

Abstract

This information processing device comprises: an estimation unit that estimates estimation information including at least one event on the basis of a real space image by a first image recognition model that recognizes an event related to a recognition object represented in an image; a virtual space configuration unit that configures, on the basis of the estimation information, a virtual recognition object that is a virtual object to be recognized that reproduces the event into a virtual space representing the real space; a virtual space image acquisition unit that acquires a virtual space image that is an image obtained by projecting the virtual space including the virtual recognition object from a viewpoint position in the virtual space; and a training data configuration unit that outputs training data of the image recognition model including the virtual space image in which the event included in the estimation information is associated.
Need to check novelty before this filing date? Find Prior Art

Description

TECHNICAL FIELD

[0001] The present invention relates to an information processing apparatus and an information processing method.

BACKGROUND ART

[0002] For training an image recognition model that recognizes a predetermined event related to a recognition target represented in an image, training data consisting of pairs of an image and an annotation indicating the correct answer of the event is required. Also, a technique is known that uses an image of a recognition target constructed in a virtual space to generate training data for training an image recognition model. For example, the following Patent Document 1 discloses a technique in which a renderer generates a composite image of a scene using a content library or the like, and the composite image is used to generate training data for a real-world navigation application.

PRIOR ART DOCUMENTS

PATENT DOCUMENTS

[0003]

PATENT DOCUMENT 1

SUMMARY OF INVENTION

PROBLEMS TO BE SOLVED BY THE INVENTION

[0004] Obtaining images for training data by imaging in the real space was time-consuming and costly. In addition, the accuracy of annotations obtained by acquiring events related to the recognition target from real-space images was low. On the other hand, annotations corresponding to images obtained by imaging a virtual space that imitates the real space have high accuracy. However, it has been difficult to comprehensively reproduce various situations of a recognition target that may occur in the real space with the recognition target constructed in the virtual space. In addition, there have been cases where images for training data are generated based on a virtual space in which situations that are unnatural in the real space are reproduced.

[0005] Therefore, this disclosure aims to obtain highly accurate training data for image recognition models. [Means for solving the problem]

[0006] To solve the above problems, an information processing device relating to one aspect of this disclosure includes: an estimation unit that estimates estimation information including at least one event based on a real-space image using a first image recognition model that recognizes events relating to a recognition object represented in an image; a virtual space configuration unit that configures a virtual recognition object, which is a virtual recognition object that reproduces the event, in a virtual space representing the real space based on the estimation information; a virtual space image acquisition unit that acquires a virtual space image, which is an image projected from a viewpoint position in the virtual space, of the virtual space including the virtual recognition object; and a training data configuration unit that outputs training data for an image recognition model, which includes a virtual space image associated with the events included in the estimation information.

[0007] Based on the above aspects, since the virtual recognition target, which reproduces the events recognized by the image recognition model, is constructed based on estimated information, a virtual space that can occur in real space is constructed. Then, since the training data is constructed from virtual space images projected from a virtual space that can occur in real space, suitable training data for training an image recognition model to recognize images in real space is obtained. Furthermore, since the virtual space images included in the training data are obtained by projecting the virtual space, higher resolution is possible compared to images optically captured from real space. Therefore, it is possible to improve the accuracy of machine learning of the image recognition model. Moreover, since the training data is constructed by annotating events based on estimated information onto virtual space images that project a virtual space containing the virtual recognition target reproduced based on estimated information, it is possible to obtain training data with extremely high accuracy as ground truth information in image recognition. [Effects of the Invention]

[0008] This makes it possible to obtain highly accurate training data for image recognition models. [Brief explanation of the drawing]

[0009] [Figure 1] This is a block diagram showing the functional configuration of the information processing device of this embodiment. [Figure 2] This figure shows the acquisition process for obtaining estimated information from real-world images. [Figure 3] This diagram schematically shows a virtual space where the person to be recognized is placed based on estimated information. [Figure 4] This diagram shows the acquisition of virtual space images captured by a virtual camera placed in a virtual space, and the structure of the training data. [Figure 5] This figure shows the process of generating an image recognition model using machine learning with training data. [Figure 6] This is a flowchart showing the processing details of information processing methods in an information processing system. [Figure 7] This is a diagram showing the structure of an information processing program. [Figure 8] This is a hard block diagram of an information processing device. [Modes for carrying out the invention]

[0010] Embodiments of the information processing apparatus according to the present invention will be described with reference to the drawings. Where possible, the same parts will be denoted by the same reference numerals, and redundant descriptions will be omitted.

[0011] Figure 1 is a block diagram showing the device configuration of an information processing system including an information processing device according to this embodiment, and the functional configuration of the information processing device. The information processing system 1 is a system that generates training data for learning an image recognition model that recognizes events related to recognition targets represented in an image, and as an example, generates training data for an image recognition model that recognizes the orientation of a person's face represented in an image of real space. The events recognized by the image recognition model are not limited to the orientation of a person's face, and for example, a person with bad behavior in an image may be the event to be recognized.

[0012] As shown in Figure 1, the information processing apparatus 10 functionally includes a real space image acquisition unit 11, an estimation unit 12, a virtual space configuration unit 13, a virtual space image acquisition unit 14, a training data configuration unit 15, and a model generation unit 16. In the example shown in Figure 1, the functional units 11 to 16 are configured in a single information processing apparatus 10, but may be configured to be distributed across a plurality of apparatuses.

[0013] Each functional unit of the information processing apparatus 10 is configured to be accessible to a storage means (storage) such as the training data storage unit 21. The training data storage unit 21 stores the generated training data. In the example shown in Figure 1, the training data storage unit 21 is configured in another apparatus accessible from the information processing apparatus 10, but may be configured in the information processing apparatus 10.

[0014] Next, each functional unit of the information processing apparatus 10 will be described. The real space image acquisition unit 11 acquires a real space image obtained by imaging the real space including the recognition target. Specifically, the real space image acquisition unit 11 acquires a real space image obtained by imaging the real space where the recognition target is present with a camera.

[0015] Figure 2 is a diagram illustrating acquisition processing for acquiring a real space image and acquiring estimation information from the real space image. For example, the real space image acquisition unit 11 acquires a real space image rp showing a person, which is obtained by imaging, with a camera rc, a real space rs such as a store where a person hr as a recognition target is present. Since the real space image rp is an optically captured image, it is composed of pixel data.

[0016] The estimation unit 12 estimates estimation information including at least one event based on the real space image, using a first image recognition model that recognizes events related to a predetermined recognition target represented in the image.

[0017] Specifically, the estimation unit 12 causes an event to be recognized based on the real space image rp, using a first image recognition model md1 that recognizes a predetermined event. The estimation unit 12 outputs estimation information ri composed of data indicating the recognized event.

[0018] For example, the first image recognition model md1 is a model that recognizes a person hr as a recognition target and recognizes the orientation of the person's face as a predetermined event. As shown in Figure 2, the estimation unit 12 uses the first image recognition model md1 to output estimation information ri including data indicating the face orientation of the person hr based on the real space image rp. The estimation information ri may include data such as yaw, pitch and roll representing the face orientation.

[0019] The virtual space configuration unit 13 configures a virtual recognition target, which is a virtual recognition target that reproduces a predetermined event, in the virtual space representing the real space based on the estimation information ri. Specifically, the virtual space configuration unit 13 uses a three-dimensional virtual space model representing the real space rs to arrange the virtual recognition target that reproduces the event indicated by the estimation information ri in the virtual space represented by the virtual space model.

[0020] Figure 3 is a diagram schematically showing a virtual space in which a person as a recognition target is arranged based on estimation information. In the example shown in Figure 3, the virtual space configuration unit 13 arranges a person hm that reproduces the face orientation represented by the estimation information ri in the virtual space vs represented by the virtual space model. The position where the person hm is arranged can be acquired by a known analysis process based on the real space image rp.

[0021] Note that the three-dimensional virtual space model may be generated by any known method. As an example, the three-dimensional virtual space model can be generated based on a two-dimensional real space image. That is, the depth of what is represented in each pixel of the two-dimensional real space image is estimated, converted into point cloud data based on the depth and color value of each pixel, and a three-dimensional virtual space is generated by three-dimensional display based on the point cloud data.

[0022] In this way, since the virtual recognition target that reproduces the event to be recognized by the image recognition model is configured based on the estimation information ri, it is possible to configure a virtual space in which situations that may occur in the real space are reproduced.

[0023] The virtual space image acquisition unit 14 acquires a virtual space image, which is an image projected from the viewpoint position within the virtual space vs, including the virtual recognition target. Specifically, the virtual space image acquisition unit 14 acquires a virtual space image by capturing images of the virtual recognition target configured in the virtual space using a virtual camera placed in the virtual space vs.

[0024] Figure 4 is a diagram showing the acquisition of virtual space images captured by a virtual camera located in the virtual space and the structure of the training data. As shown in Figure 4, the virtual space image acquisition unit 14 acquires a virtual space image vp by capturing images of the virtual space vs where the person hm to be recognized is located, using a virtual camera vc located in the virtual space vs.

[0025] The virtual space image acquisition unit 14 may acquire a virtual space image vp by capturing images of the virtual space vs reproduced by a virtual space model for representing the real space rs. In this case, the virtual recognition target is placed in the virtual space vs constructed by the model, and a virtual space image can be acquired by capturing images of the virtual space including the virtual recognition target. Therefore, a virtual space image with higher resolution can be acquired compared to an image captured by optical means.

[0026] Furthermore, the virtual space image acquisition unit 14 may acquire a virtual space image vp by capturing a virtual space in which the virtual recognition target is constructed with the real space image rp as the background. That is, by setting the position of the virtual camera vc in the virtual space vs to the same position of the camera rc in the real space rs when acquiring the real space image rp, the real space image rp can be used as the background other than the virtual recognition target in the virtual space image vp. In this case, the virtual space image acquisition unit 14 may acquire a virtual space image vp by compositing the real space image rp as the background onto the image of the virtual space in which the recognition target is placed. This reduces the processing load required to construct a model of the virtual space for acquiring the virtual space image.

[0027] Furthermore, the position of the virtual camera vc in the virtual space vs reproduced by the virtual space model does not need to be the same as the position of the camera rc in the real space rs. If the position of the virtual camera vc in the virtual space vs is the same as the position of the camera rc in the real space rs, as mentioned above, it is possible to use the real space image rs as the background in the virtual space image vp, and suitable training data can be obtained for training an image recognition model that recognizes the image in the real space. On the other hand, even if the position of the virtual camera vc in the virtual space vs is different from the position of the camera rc in the real space rs, suitable training data can be obtained because the position of the virtual recognition target (e.g., person hp) in the virtual space vs reflects the position of person hr in the real space rs.

[0028] The training data structuring unit 15 outputs training data for an image recognition model, which includes virtual space images associated with events included in the estimated information ri. Specifically, as shown in Figure 4, the training data structuring unit 15 generates training data td for supervised machine learning by associating the virtual space images vp acquired by the virtual space image acquisition unit 14 with the data indicating the events to be recognized in image recognition, which are included in the estimated information ri, as annotation an (ground truth information). In this embodiment, training data for an image recognition model that recognizes the orientation of a person's face is generated, so the annotation an is data such as yaw, pitch, and roll that represent the orientation of the face.

[0029] The training data structuring unit 15 then outputs the generated training data td. Specifically, the training data structuring unit 15 may store the generated training data td in the training data storage unit 21.

[0030] In this way, since the virtual space image vp, which projects a virtual space vs containing a virtual recognition target reproduced based on estimated information ri, is annotated with events based on the estimated information ri to construct the training data td, it becomes possible to obtain training data with extremely high accuracy as ground truth information in image recognition.

[0031] The model generation unit 16 generates a second image recognition model by machine learning using the training data td. Figure 5 shows the process of generating an image recognition model by machine learning using the training data td. As shown in Figure 5, the model generation unit 16 acquires training data td from the training data storage unit 21, which consists of pairs of virtual space images vp representing the recognition target that reproduces the predetermined event to be recognized, and annotations an consisting of data indicating the predetermined event. Using the acquired training data td, the model generation unit 16 performs machine learning on the second image recognition model md2. The model generation unit 16 generates a second image recognition model md2 that can recognize the predetermined event in the recognition target with high accuracy by machine learning using the training data td.

[0032] Figure 6 is a flowchart showing the processing details of an information processing method that generates training data for learning an image recognition model that recognizes events related to recognition targets represented in an image in the information processing device 10, and generates an image recognition model using machine learning with the training data.

[0033] In step S1, the real-space image acquisition unit 11 acquires a real-space image rp, which is an image of the real-space rs that includes the recognition target (person hr). Specifically, the real-space image acquisition unit 11 acquires a real-space image rp, which is an image of the real-space where the recognition target exists, captured by the camera rc.

[0034] In step S2, the estimation unit 12 estimates estimated information ri, which includes at least one event, based on the real-space image rp, using a first image recognition model md1 that recognizes events related to a predetermined recognition object represented in the image.

[0035] In step S3, the virtual space configuration unit 13 configures a virtual recognition target (person hm), which is a virtual recognition target that reproduces a predetermined event, in the virtual space vs representing the real space, based on the estimated information ri.

[0036] In step S4, the virtual space image acquisition unit 14 acquires a virtual space image vp, which is an image projected from the viewpoint position within the virtual space vs that includes the virtual recognition target.

[0037] In step S5, the training data structuring unit 15 constructs and outputs training data td, which includes virtual space images vp associated with events included in the estimated information ri.

[0038] In step S6, the model generation unit 16 generates a second image recognition model md2 using machine learning with the training data td.

[0039] Next, with reference to Figure 7, an information processing program for causing a computer to function as the information processing device 10 of this embodiment will be described. Figure 7 is a diagram showing the configuration of the information processing program. The information processing program P1 is composed of a main module m10 that comprehensively controls information processing in the information processing device 10, a real-world image acquisition module m11, an estimation module m12, a virtual space configuration module m13, a virtual space image acquisition module m14, a training data configuration module m15, and a model generation module m16. Each of the modules m11 to m16 realizes the respective functions for each functional unit 11 to 16.

[0040] The information processing program P1 may be transmitted via a transmission medium such as a communication line, or it may be stored in a recording medium M1, as shown in Figure 7.

[0041] According to the information processing system 1, information processing device 10, information processing method, and information processing program P1 of this embodiment described above, a virtual recognition target in which the events to be recognized by the image recognition model are reproduced is constructed based on the estimated information ri, thereby constructing a virtual space that can occur in real space. Then, by constructing training data td using a virtual space image vp projected with the virtual space vs that can occur in real space, training data td suitable for training an image recognition model to recognize images in real space is obtained. Furthermore, since the virtual space image vp included in the training data td is obtained by projecting the virtual space vs, it is possible to achieve higher resolution compared to images optically captured in real space. Therefore, it is possible to improve the accuracy of machine learning of the image recognition model. Moreover, since the training data td is constructed by annotating events based on the estimated information ri onto the virtual space image vp, which is a projection of the virtual space vs that includes the virtual recognition target reproduced based on the estimated information ri, it is possible to obtain training data td with extremely high accuracy as correct information in image recognition.

[0042] The information processing apparatus and information processing method relating to this disclosure may have the following configurations. The operation and effects of each configuration are described below.

[0043] An information processing device relating to one aspect of this disclosure comprises: an estimation unit that estimates estimation information including at least one event based on a real-space image using a first image recognition model that recognizes events relating to a recognition object represented in an image; a virtual space configuration unit that configures a virtual recognition object, which is a virtual recognition object that reproduces the event, in a virtual space representing the real space based on the estimation information; a virtual space image acquisition unit that acquires a virtual space image, which is an image projected from a viewpoint position in the virtual space, of the virtual space including the virtual recognition object; and a training data configuration unit that outputs training data for an image recognition model, which includes a virtual space image associated with the events included in the estimation information.

[0044] An information processing method relating to one aspect of this disclosure includes: an estimation step in which a first image recognition model, executed by a processor and recognizing events relating to a recognition object represented in an image, estimates estimated information including at least one event based on a real-space image captured of a real-space containing the recognition object; a virtual-space configuration step in which a virtual recognition object, which is a virtual recognition object that reproduces the event, is constructed in the virtual space based on the estimated information; a virtual-space image acquisition step in which a virtual-space image is obtained, which is an image projected of the virtual space containing the virtual recognition object from a viewpoint position in the virtual space; and a training data configuration step in which training data for the image recognition model is output, which includes a real-space image associated with the event included in the estimated information.

[0045] Based on the above aspects, since the virtual recognition target, which reproduces the events recognized by the image recognition model, is constructed based on estimated information, a virtual space that can occur in real space is constructed. Then, since the training data is constructed from virtual space images projected from a virtual space that can occur in real space, suitable training data for training an image recognition model to recognize images in real space is obtained. Furthermore, since the virtual space images included in the training data are obtained by projecting the virtual space, higher resolution is possible compared to images optically captured from real space. Therefore, it is possible to improve the accuracy of machine learning of the image recognition model. Moreover, since the training data is constructed by annotating events based on estimated information onto virtual space images that project a virtual space containing the virtual recognition target reproduced based on estimated information, it is possible to obtain training data with extremely high accuracy as ground truth information in image recognition.

[0046] Furthermore, the information processing device relating to other aspects may further include a model generation unit that generates a second image recognition model by machine learning using training data.

[0047] Based on the above aspects, by performing machine learning using training data that has high accuracy in correct information for image recognition and is suitable for recognizing images of real space that include the object to be recognized, it becomes possible to obtain an image recognition model that can recognize events related to the object to be recognized with high accuracy.

[0048] Furthermore, in information processing devices relating to other aspects, the virtual space image acquisition unit may acquire a virtual space image by capturing images of a virtual recognition target configured in the virtual space using a virtual camera placed in the virtual space.

[0049] Based on the above aspects, since virtual space images including virtual recognition targets can be obtained through data processing without relying on optical imaging of real space, training data can be easily constructed.

[0050] Furthermore, in information processing devices relating to other aspects, the virtual space image acquisition unit may acquire a virtual space image by capturing a virtual space in which a virtual recognition target is constructed with a real space image as the background.

[0051] Based on the above aspects, by making the position of the virtual camera (viewpoint) in the virtual space the same as the camera position in the real space when capturing images in the real space, the real space image can be used as the background for the virtual recognition target in the virtual space image. This reduces the processing load required to construct the virtual space model for acquiring the virtual space image.

[0052] Furthermore, in information processing devices relating to other aspects, the virtual space image acquisition unit may acquire a virtual space image by capturing images of a virtual space in which a virtual recognition target is placed within a virtual space model that reproduces the real space.

[0053] Based on the above aspects, a virtual recognition target is placed in a virtual space constructed as a model, and a high-resolution virtual space image can be obtained by imaging the virtual space including the virtual recognition target.

[0054] Furthermore, in information processing devices relating to other aspects, the object of recognition may be a person, the event may be the orientation of the person's face, and the estimated information may include information indicating the orientation of the person's face.

[0055] Based on the above aspects, it becomes possible to obtain training data for generating an image recognition model that can accurately recognize the orientation of a person's face.

[0056] The block diagram shown in Figure 1 represents functional units. These functional blocks (components) are implemented by any combination of at least one of hardware and software. Furthermore, the method of implementing each functional block is not particularly limited. That is, each functional block may be implemented using one device that is physically or logically coupled, or it may be implemented using two or more physically or logically separated devices that are directly or indirectly connected (for example, using wired or wireless connections). A functional block may also be implemented by combining the above one device or the above multiple devices with software.

[0057] Functions include, but are not limited to, judgment, decision, judgment, calculation, calculation, processing, derivation, investigation, exploration, confirmation, reception, transmission, output, access, resolution, selection, selection, establishment, comparison, assumption, expectation, assumption, broadcasting, notifying, communicating, forwarding, configuring, reconfiguring, allocating (mapping), and assigning. For example, a functional block (configuration part) that enables transmission is called a transmitting unit or transmitter. As mentioned above, the method of implementation is not particularly limited.

[0058] For example, the information processing device 10 in one embodiment of the present invention may function as a computer. Figure 8 shows an example of the hardware configuration of the information processing device 10 according to this embodiment. Physically, the information processing device 10 may be configured as a computer device including a processor 1001, memory 1002, storage 1003, communication device 1004, input device 1005, output device 1006, bus 1007, etc.

[0059] In the following explanation, the term "device" can be replaced with "circuit," "device," "unit," etc. The hardware configuration of the information processing device 10 may include one or more of the devices shown in Figure 8, or it may be configured to omit some of the devices.

[0060] Each function in the information processing device 10 is realized by loading predetermined software (programs) onto hardware such as the processor 1001 and memory 1002, allowing the processor 1001 to perform calculations and control communication by the communication device 1004, as well as the reading and / or writing of data to the memory 1002 and storage 1003.

[0061] The processor 1001 controls the entire computer, for example, by running the operating system. The processor 1001 may consist of a central processing unit (CPU) that includes interfaces with peripheral devices, control units, arithmetic units, registers, etc. For example, the various functional units 11 to 16 shown in Figure 1 may be implemented by the processor 1001.

[0062] Furthermore, the processor 1001 reads programs (program code), software modules, and data from the storage 1003 and / or communication device 1004 into the memory 1002 and executes various processes accordingly. The program used is one that causes the computer to execute at least a part of the operations described in the above embodiment. For example, each of the functional units 11 to 16 of the information processing device 10 may be stored in the memory 1002 and implemented by a control program that runs on the processor 1001. Although the above-described processes have been explained as being executed by one processor 1001, they may be executed simultaneously or sequentially by two or more processors 1001. The processor 1001 may be implemented on one or more chips. The program may also be transmitted from a network via a telecommunications line.

[0063] Memory 1002 is a computer-readable recording medium and may consist of at least one of the following: ROM (Read Only Memory), EPROM (Erasable Programmable ROM), EEPROM (Electrically Erasable Programmable ROM), RAM (Random Access Memory), etc. Memory 1002 may also be called a register, cache, main memory, etc. Memory 1002 can store executable programs (program code), software modules, etc., for carrying out an information processing method according to one embodiment of the present invention.

[0064] The storage 1003 is a computer-readable recording medium and may consist of at least one of the following: an optical disc such as a CD-ROM (Compact Disc ROM), a hard disk drive, a flexible disk, a magneto-optical disk (e.g., a compact disc, a digital multipurpose disc, a Blu-ray® disc), a smart card, flash memory (e.g., a card, a stick, a key drive), a floppy® disk, a magnetic strip, etc. The storage 1003 may also be called an auxiliary storage device. The above-mentioned storage medium may be, for example, a database, server, or other suitable medium including memory 1002 and / or storage 1003.

[0065] The communication device 1004 is hardware (transceiver / receiver device) for communicating between computers via a wired and / or wireless network, and is also referred to as a network device, network controller, network card, communication module, etc.

[0066] The input device 1005 is an input device that accepts input from an external source (e.g., a keyboard, mouse, microphone, switch, button, sensor, etc.). The output device 1006 is an output device that outputs to an external source (e.g., a display, speaker, LED lamp, etc.). The input device 1005 and the output device 1006 may be configured as an integrated unit (e.g., a touch panel).

[0067] Furthermore, each device, such as the processor 1001 and the memory 1002, is connected by a bus 1007 for communicating information. The bus 1007 may consist of a single bus or different buses may be used for communication between devices.

[0068] Furthermore, the information processing device 10 may be configured to include hardware such as a microprocessor, a digital signal processor (DSP), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), and an FPGA (Field Programmable Gate Array), and some or all of each functional block may be realized by such hardware. For example, the processor 1001 may be implemented using at least one of these pieces of hardware.

[0069] Information notification is not limited to the embodiments described herein and may be carried out by other means. For example, information notification may be carried out by physical layer signaling (e.g., DCI (Downlink Control Information), UCI (Uplink Control Information)), upper layer signaling (e.g., RRC (Radio Resource Control) signaling, MAC (Medium Access Control) signaling, broadcast information (MIB (Master Information Block), SIB (System Information Block))), other signals, or combinations thereof. RRC signaling may also be called RRC messages, and may be, for example, RRC Connection Setup messages, RRC Connection Reconfiguration messages, etc.

[0070] Each aspect / embodiment described in this disclosure may be applied to at least one of the following systems: LTE (Long Term Evolution), LTE-A (LTE-Advanced), SUPER 3G, IMT-Advanced, 4G (4th generation mobile communication system), 5G (5th generation mobile communication system), FRA (Future Radio Access), NR (new Radio), W-CDMA (registered trademark), GSM (registered trademark), CDMA2000, UMB (Ultra Mobile Broadband), IEEE 802.11 (Wi-Fi (registered trademark)), IEEE 802.16 (WiMAX (registered trademark)), IEEE 802.20, UWB (Ultra-WideBand), Bluetooth (registered trademark), and other appropriate systems, as well as next-generation systems extended based thereon. Furthermore, multiple systems may be applied in combination (for example, a combination of at least one of LTE and LTE-A with 5G).

[0071] The processing procedures, sequences, flowcharts, etc., of each aspect / embodiment described herein may be reordered, provided they are consistent with each other. For example, the methods described herein present various step elements in an exemplary order and are not limited to that specific order.

[0072] The specific operations described in this disclosure as being performed by a base station may, in some cases, be performed by its upper node. In a network consisting of one or more network nodes having a base station, it is clear that various operations performed for communication with a terminal can be performed by the base station and at least one other network node (for example, an MME or S-GW, but not limited to these). Although the above example illustrates a case where there is one other network node besides the base station, it may also be a combination of multiple other network nodes (for example, an MME and an S-GW).

[0073] Information can be output from a higher layer (or lower layer) to a lower layer (or higher layer). Input and output may also occur via multiple network nodes.

[0074] Input and output information may be stored in a specific location (e.g., memory) or managed in a management table. Input and output information may be overwritten, updated, or appended to. Output information may be deleted. Input information may be sent to other devices.

[0075] The determination may be made by a value represented by 1 bit (0 or 1), by a boolean value (true or false), or by a numerical comparison (for example, a comparison with a predetermined value).

[0076] Each aspect / embodiment described herein may be used individually, in combination, or switched between as needed during implementation. Furthermore, notification of specific information (e.g., notification that "X is") is not limited to explicit notification, but may also be implicit (e.g., by not providing such notification).

[0077] Although the present disclosure has been described in detail above, it will be clear to those skilled in the art that the present disclosure is not limited to the embodiments described herein. The present disclosure can be implemented in modified and altered forms without departing from the intent and scope of the present disclosure as defined by the claims. Therefore, the descriptions in the present disclosure are illustrative and not intended to be restrictive in any way.

[0078] Software should be broadly interpreted to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executable files, execution threads, procedures, functions, and so on, whether they are called software, firmware, middleware, microcode, hardware description languages, or by any other name.

[0079] Furthermore, software, instructions, etc., may be transmitted and received via a transmission medium. For example, if software is transmitted from a website, server, or other remote source using wired technologies such as coaxial cable, fiber optic cable, twisted pair, and digital subscriber lines (DSL) and / or wireless technologies such as infrared, radio, and microwave, these wired and / or wireless technologies are included in the definition of a transmission medium.

[0080] The information, signals, etc. described in this disclosure may be represented using any of the various different techniques. For example, the data, instructions, commands, information, signals, bits, symbols, chips, etc. that may be referred to throughout the above description may be represented by voltage, current, electromagnetic waves, magnetic fields or magnetic particles, optical fields or photons, or any combination thereof.

[0081] In addition, terms described in this disclosure and / or terms necessary for understanding this specification may be replaced with terms having the same or similar meaning.

[0082] The terms “system” and “network” as used in this disclosure are interchangeable.

[0083] Furthermore, the information, parameters, etc., described in this disclosure may be expressed as absolute values, relative values ​​from a given value, or by corresponding other information. For example, wireless resources may be indicated by an index.

[0084] The names used for the parameters described above are not restrictive in any way. Furthermore, the formulas and other expressions using these parameters may differ from those expressly disclosed in this disclosure. Various channels (e.g., PUCCH, PDCCH, etc.) and information elements can be identified by any suitable name, and therefore, the various names assigned to these various channels and information elements are not restrictive in any way.

[0085] As used in this disclosure, the terms “determining” and “determining” may encompass a wide variety of actions. “Determining” may include, for example, judging, calculating, computing, processing, deriving, investigating, looking up, searching, inquiry (e.g., searching in a table, database, or other data structure), and ascertaining. “Determining” may also include, for example, receiving (e.g., receiving information), transmitting (e.g., sending information), input, output, and accessing (e.g., accessing data in memory). Furthermore, "judgment" and "decision" can include considering something as having been "judged" or "decided" after resolving, selecting, choosing, establishing, comparing, etc. In other words, "judgment" and "decision" can include considering something as having been "judged" or "decided" after some action. Also, "judgment (decision)" can be reinterpreted as "assuming," "expecting," or "considering."

[0086] As used in this disclosure, the phrase "based on" does not mean "based solely on" unless otherwise specified. In other words, the phrase "based on" means both "based solely on" and "based at least on."

[0087] Where the terms “first,” “second,” etc., are used in this disclosure, no reference to those elements shall generally limit the quantity or order of those elements. These terms may be used herein as a convenient way to distinguish between two or more elements. Accordingly, references to the first and second elements shall not imply that only two elements may be employed therein, or that the first element must precede the second element in any way.

[0088] In the configuration of each of the above devices, "means" may be replaced with "part," "circuit," "device," etc.

[0089] To the extent that “include,” “including,” and their variations are used herein or in the claims, these terms are intended to be inclusive, as is the term “comprising.” Furthermore, the term “or” as used herein or in the claims is not intended to be exclusive OR.

[0090] In this disclosure, if articles are added through translation, such as a, an, and the in English, this disclosure may include the fact that the noun following these articles is plural.

[0091] In this disclosure, the term "A and B are different" may mean "A and B are different from each other." The term may also mean "A and B are each different from C." Terms such as "separate" and "combine" may be interpreted similarly to "different."

[0092] The information processing apparatus 10 and information processing method of this disclosure may have the following configurations. [1] An estimation unit that estimates estimation information including at least one of the events based on the real-space image, using a first image recognition model that recognizes events related to a recognition object represented in an image, A virtual space configuration unit configures a virtual recognition target, which is a virtual recognition target that reproduces the event based on the estimated information, in a virtual space representing the real space, A virtual space image acquisition unit acquires a virtual space image which is an image projected of the virtual space including the virtual recognition target from a viewpoint position within the virtual space. A training data configuration unit that outputs training data for the image recognition model, including the virtual space image associated with the events included in the estimation information, An information processing device equipped with the following features. [2] The system further comprises a model generation unit that generates a second image recognition model using machine learning with the aforementioned training data. [1] The information processing device described above. [3] The virtual space image acquisition unit acquires the virtual space image by capturing the virtual recognition target configured in the virtual space using a virtual camera placed in the virtual space. The information processing device described in [1] or [2]. [4] The virtual space image acquisition unit acquires the virtual space image by capturing the virtual space in which the virtual recognition target is constructed with the real space image as the background. [3] The information processing device described above. [5] The virtual space image acquisition unit acquires the virtual space image by capturing the virtual space in which the virtual recognition target is placed within a virtual space model that reproduces the real space. [3] The information processing device described above. [6] The object of recognition is a person, The aforementioned event is the direction of the person's face, The aforementioned estimation information includes information indicating the direction of the person's face. An information processing device as described in any one of items [1] to [5]. [7] An estimation step in which a first image recognition model that recognizes events relating to a recognized object represented in an image estimates estimated information including at least one of the events based on a real-space image captured of the real-space including the recognized object, A virtual space configuration step in which a virtual recognition target, which is a virtual recognition target that reproduces the event based on the estimated information, is configured in a virtual space, A virtual space image acquisition step involves acquiring a virtual space image, which is an image projected of the virtual space including the virtual recognition target from a viewpoint position within the virtual space. A training data configuration step that outputs training data for the image recognition model, including the real-space image associated with the event included in the estimation information, An information processing method performed by a processor, comprising: [Explanation of symbols]

[0093] 1... Information processing system, 10... Information processing device, 11... Real-world image acquisition unit, 12... Estimation unit, 13... Virtual space configuration unit, 14... Virtual space image acquisition unit, 15... Training data configuration unit, 16... Model generation unit, 21... Training data storage unit, M1... Recording medium, m11... Real-world image acquisition module, m12... Estimation module, m13... Virtual space configuration module, m14... Virtual space image acquisition module, m15... Training data configuration module, m16... Model generation module, md1... First image recognition model, md2... Second image recognition model, P1... Information processing program, rc... Camera, ri... Estimation information, rp... Real-world image, td... Training data, vc... Virtual camera, vs... Virtual space.

Claims

1. An estimation unit that estimates estimation information including at least one of the aforementioned events based on a real-space image captured of the real-space including the recognized object, using a first image recognition model that recognizes events related to a recognized object represented in an image, A virtual space configuration unit configures a virtual recognition target, which is a virtual recognition target that reproduces the event based on the estimated information, in a virtual space representing the real space, A virtual space image acquisition unit acquires a virtual space image which is an image projected of the virtual space including the virtual recognition target from a viewpoint position within the virtual space. A training data configuration unit generates and outputs training data for the image recognition model by associating the data indicating the events included in the estimation information with the virtual space image as annotations, An information processing device equipped with the following features.

2. The system further comprises a model generation unit that generates a second image recognition model using machine learning with the aforementioned training data. The information processing apparatus according to claim 1.

3. The virtual space image acquisition unit acquires the virtual space image by capturing the virtual recognition target configured in the virtual space using a virtual camera placed in the virtual space. The information processing apparatus according to claim 1.

4. The virtual space image acquisition unit acquires the virtual space image by capturing the virtual space in which the virtual recognition target is constructed with the real space image as the background. The information processing apparatus according to claim 3.

5. The virtual space image acquisition unit acquires the virtual space image by capturing the virtual space in which the virtual recognition target is placed within a virtual space model that reproduces the real space. The information processing apparatus according to claim 3.

6. The object of recognition is a person, The aforementioned event is the direction of the person's face, The aforementioned estimation information includes information indicating the direction of the person's face. The information processing apparatus according to claim 1.

7. An estimation step in which a first image recognition model that recognizes events relating to a recognized object represented in an image estimates estimated information including at least one of the events based on a real-space image captured of the real-space including the recognized object, A virtual space configuration step in which a virtual recognition target, which is a virtual recognition target that reproduces the event based on the estimated information, is configured in a virtual space, A virtual space image acquisition step involves acquiring a virtual space image, which is an image projected of the virtual space including the virtual recognition target from a viewpoint position within the virtual space. A training data configuration step in which training data for the image recognition model is generated and output by associating the data indicating the event included in the estimation information with the virtual space image as annotations, An information processing method performed by a processor, comprising:

Citation Information

Patent Citations

  • Recognition model distribution system and recognition model updating method

    JP2021043622A

  • Unsupervized learning of scene structure for synthetic data generation

    JP2021140767A

  • Learning data creation method, learning data creation device, and program

    JP2022024189A