Model generation device, model generation method and program

Abstract

A model generation device 200 according to the present disclosure comprises: a detection means 201 that detects, using an object detection model, a first position that is an object position in a first image and a second position that is an object position in a second image that is different in angular field from the first image; a generation means 202 that generates from the first position a corresponding position that corresponds, in the second image, to the first position, on the basis of the difference in angular field between the first and second images; and a training means 203 that trains the object detection model on the basis of the second position and the corresponding position.

Description

Model generation device, model generation method, and program

[0001] The present disclosure relates to a model generation device that generates a model for detecting an object included in an image.

[0002] Techniques for detecting objects from photographed images of the objects are known. For example, as described in Patent Literature 1, a system has been proposed that captures images of store shelves, identifies product positions, and performs shelf planogram analysis. In such a system, an object detection model for identifying product positions is trained in advance using a large number of photographed images of the shelves. During operation, the trained object detection model is used to identify the positions of products included in images of the shelves photographed in each store.

[0003] Here, Patent Document 1 points out the problem that captured images of shelves on which products are displayed are affected by the environment, such as the angle of view at the time of capture, which can result in misrecognition or missed recognition of products, resulting in a decrease in recognition accuracy.In response to this problem, Patent Document 1 describes a method for detecting areas where there is a high possibility of missed recognition using information about fixtures.

[0004] Japanese Patent Application Laid-Open No. 2020-61158

[0005] However, the method described in Patent Document 1 requires that information about fixtures be stored in advance, and if such information is not available, it is not possible to detect missed products. Therefore, the problem of the accuracy of object detection in an image decreasing depending on the angle of view of the image still remains.

[0006] An object of the present disclosure is to solve the above-mentioned problem that the accuracy of object detection in an image decreases depending on the angle of view at which the image is captured.

[0007] A model generation device according to one embodiment of the present disclosure includes: a detection means for detecting, using an object detection model, a first position that is the position of an object in a first image and a second position that is the position of the object in a second image that has a different angle of view from the first image; a generation means for generating, from the first position, a corresponding position that is a position in the second image that corresponds to the first position, based on a difference in angle of view between the first image and the second image; and a learning means for learning the object detection model based on the second position and the corresponding position.

[0008] Furthermore, a model generation method according to one embodiment of the present disclosure includes the following configuration: using an object detection model, detect a first position, which is the position of an object in a first image, and a second position, which is the position of the object in a second image having a different angle of view from the first image; before and after detecting the second position, generate a corresponding position, which is a position in the second image corresponding to the first position, from the first position based on the difference in angle of view between the first image and the second image; and learning the object detection model based on the second position and the corresponding position.

[0009] Furthermore, a program according to one embodiment of the present disclosure has a configuration that causes a computer to execute the following processes: using an object detection model, detect a first position that is the position of an object in a first image, and a second position that is the position of the object in a second image that has a different angle of view from the first image; generate a corresponding position that is a position in the second image that corresponds to the first position from the first position based on the difference in angle of view between the first image and the second image; and learn the object detection model based on the second position and the corresponding position.

[0010] With the above-described configuration, the present disclosure can prevent the accuracy of detecting an object in an image from decreasing due to the angle of view of the image being captured.

[0011] FIG. 1 is a diagram showing the overall configuration of an object detection system in a first embodiment of the present disclosure. FIG. 2 is a diagram showing an example of a store environment in which the object detection device disclosed in FIG. 1 is used. FIG. 3 is a block diagram showing the hardware configuration of the object detection device disclosed in FIG. 1. FIG. 4 is a block diagram showing the configuration of the object detection device disclosed in FIG. 1. FIG. 5 is a diagram showing the state of image processing by the object detection device disclosed in FIG. 1. FIG. 6 is a flowchart showing the operation during learning processing of an object detection model by the object detection device disclosed in FIG. 1. FIG. 7 is a block diagram showing the configuration of a model generation device in a second embodiment of the present disclosure.

[0012] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. <First Embodiment> [Overall Configuration] FIG. 1 shows the overall configuration of an object detection system according to a first embodiment. As shown in FIG. 1, the object detection system includes an object detection device 100 and an image database (hereinafter, "database" will be referred to as "DB") 2. The object detection device 100 acquires image data from the image DB 2 and performs object detection. Accordingly, the object detection device 100 of the present disclosure also functions as a model generation device that generates, through learning, an object detection model to be used in object detection. When the object detection device 100 learns the object detection model, a learning dataset is stored in the image DB 2. On the other hand, when the object detection device 100 is used in an actual store, i.e., when inferring the detection of an object from an image, images taken in the store are stored in the image DB 2.

[0013] [Example of Store Environment] Fig. 2 shows an example of a store environment in which the object detection device 100 is used. Product shelves 3 are installed in the store, and various products are displayed on the product shelves 3. Surveillance cameras 4 are installed in the store and capture images of the product shelves 3. The images captured by the surveillance cameras 4 are sent to a terminal device 6 and stored in an image DB 2 connected to the terminal device 6. A store clerk stands in front of the product shelves 3 and captures a front image of the product shelves 3 using a camera 5 on a mobile device. The image captured by the camera 5 on the mobile device is sent to the terminal device 6 and recorded in the image DB 2. The object detection device 100 is realized by, for example, the terminal device 6 or another terminal device.

[0014] 3 is a block diagram showing the hardware configuration of the object detection device 100. As shown in the figure, the object detection device 100 includes a communication unit 101, a processor 102, a memory 103, and a recording medium 104.

[0015] The communication unit 101 communicates with the image DB 3 via a wired or wireless connection, and acquires a training data set prepared in advance, images captured by the store's cameras 4, etc. The processor 102 is a computer such as a CPU (Central Processing Unit), and controls the entire object detection device 100 by executing a program prepared in advance. The processor 102 may be a graphics processing unit (GPU), a field-programmable gate array (FPGA), a digital signal processor (DSP), a micro processing unit (MPU), a floating point number processing unit (FPU), a physics processing unit (PPU), a tensor processing unit (TPU), a quantum processor, a microcontroller, or a combination thereof. Specifically, the processor 102 executes a pre-learning process and an additional learning process, which will be described later.

[0016] The memory 103 is configured by a ROM (Read Only Memory), a RAM (Random Access Memory), etc. The memory 103 is also used as a working memory while the processor 102 is executing various processes.

[0017] The recording medium 104 is a non-volatile, non-transitory recording medium such as a disk-shaped recording medium or a semiconductor memory, and is configured to be detachable from the object detection device 100. The recording medium 104 records various programs to be executed by the processor 102. When the object detection device 100 executes various processes, the programs recorded on the recording medium 104 are loaded into the memory 103 and executed by the processor 102.

[0018] [Configuration of Object Detection Device] Next, the configuration of the object detection device 100 will be described. As shown in FIG. 4 , the object detection device 100, to which the above-described image DB 2 is connected, includes an object position estimation unit 20, a loss calculation unit 30, a geometric deformation estimation unit 50, and an automatic correction assignment unit 60. The object position estimation unit 20, the loss calculation unit 30, the geometric deformation estimation unit 50, and the automatic correction assignment unit 60 can be realized by the processor 102 executing a program stored in the memory 103 or the recording medium 104. In this case, the geometric deformation estimation unit 50 further includes a feature point extraction unit 51, a feature point coincidence calculation unit 52, and a geometric deformation parameter calculation unit 53. The automatic correction assignment unit 60 further includes a detection result transfer unit 61 and a correction rectangle generation unit 62. Each component will be described in detail below.

[0019] First, the function of pre-learning performed by the object position estimation unit 20 and the loss calculation unit 30 will be described. Pre-learning is a process of generating a basic object detection model. For this reason, a pre-learning image dataset used in pre-learning is first stored in the image DB 2. Specifically, the pre-learning image dataset includes pre-learning images of pre-prepared product shelves and ground truth data for product positions. For example, the pre-learning images are images of product shelves photographed from the front, and the ground truth data is represented by the coordinates of the vertices of a rectangle that indicates the position of an object included in each pre-learning image.

[0020] The object position estimation unit 20 detects an object included in an input image using an object detection model. Specifically, during pre-learning, the object position estimation unit 20 uses the object detection model to estimate rectangular coordinates indicating the position of an object included in a pre-learning image input from the image DB 2. The object detection model is configured by a neural network using, for example, a CNN (Convolutional Neural Network). The object position estimation unit 20 outputs to the loss calculation unit 30 the rectangular coordinates estimated from the pre-learning image input from the image DB 2 and correct answer data of the object position included in the pre-learning image associated with the input pre-learning image.

[0021] The loss calculation unit 30 calculates the loss using the input ground truth data for the object position and the estimation result by the object position estimation unit 20. Specifically, the loss calculation unit 30 calculates the error between the rectangular coordinates of the object position included in the input ground truth data and the rectangular coordinates that are the estimation result of the object position in the pre-learning image by the object position estimation unit 20, and sets this as the loss. The loss calculation unit 30 then updates the parameters of the object detection model of the object position estimation unit 20 so as to reduce the calculated loss. In this way, the parameters of the object detection model are updated until the loss value converges to a predetermined value or less, and pre-learning of the object detection model ends when the loss value converges. The object detection model at the time learning ends is obtained as the pre-trained object detection model.

[0022] As described above, the object detection model generated by pre-learning is trained on pre-learning images of product shelves primarily photographed from the front. Therefore, while the object detection accuracy for images of product shelves photographed from the front is expected to be high, the object detection accuracy for images with a different angle of view from the front image of the product shelves, such as images photographed by a store surveillance camera 4 as shown in FIG. 2 , is expected to be lower. For this reason, the object detection device 100 disclosed herein has a function for additionally training the object detection model so as to increase the object detection accuracy for images with a different angle of view from the front image, such as images photographed by the surveillance camera 4. The object detection model generated by the above-described pre-learning does not necessarily have to be generated by the object detection device 100; it may be generated by another device or prepared in advance. Below, the configuration for additional learning by the object detection device 100 is described.

[0023] For additional learning purposes, the image DB 2 includes two pairs of images (image pairs) of the same product shelf 3, i.e., the same object, captured by the surveillance camera 4 and the mobile device camera 5 at the same time within an area where there is no product movement. Here, the mobile device camera 5 captures an image of the product shelf 3 from the front, and such an image is referred to as a "front image" (first image). However, the front image does not necessarily have to be one captured strictly from the front of the product shelf 3, as long as it is captured from a nearly frontal angle. Furthermore, the surveillance camera 4 is installed, for example, on the ceiling or wall of a store, and the angle of view of the image captured by the surveillance camera 4 will be different from the front image. The image captured by the surveillance camera 4 is referred to as a "surveillance camera image" (second image). Note that the front image described in this embodiment is not necessarily an image of the product shelf 3 captured from the front by the mobile device camera 5, but may be an image captured from any direction using any type of imaging device. Furthermore, the surveillance camera image is not necessarily an image captured by the surveillance camera 4, but may be an image captured from any direction using any type of imaging device. However, the first image corresponding to the front image and the second image corresponding to the surveillance camera image are images with mutually different angles of view.

[0024] The geometric transformation estimation unit 50 (estimation means) estimates geometric transformation parameters between two images using the above-mentioned image pair for additional learning, i.e., a pair of a front image and a surveillance camera image, contained in the image DB 2. In particular, in this embodiment, the geometric transformation estimation unit 50 estimates affine transformation parameters for matching the angle of view of the camera 5 of the mobile terminal with the angle of view of the surveillance camera 4.

[0025] Specifically, the feature point extraction unit 51 extracts feature points from each of the two input images, the surveillance camera image and the front image. The extracted feature points are input to the feature point coincidence calculation unit 52 with their coordinate values ​​and feature amounts held as vectors.

[0026] The feature point coincidence calculation unit 52 calculates the similarity between the feature points of the two images extracted by the feature point extraction unit 51 and outputs pairs of feature points with high similarity. For example, for each feature point of the front image, the cosine similarity with all feature points of the surveillance camera image is calculated, and the point with the highest similarity among points with similarity higher than a predetermined value is adopted as a paired point. In other words, the feature point coincidence calculation unit 52 extracts pairs of feature points in the front image (first feature points) and corresponding feature points in the surveillance camera image (second feature points). The feature point coincidence calculation unit 52 then outputs the coordinate values ​​of each of the adopted paired points to the geometric transformation parameter calculation unit 53.

[0027] The geometric transformation parameter calculation unit 53 calculates affine transformation parameters for matching the angle of view of the front image with the angle of view of the surveillance camera image, using the coordinates of the feature point pairs that match between the two images adopted by the feature point coincidence calculation unit 52. Specifically, for each feature point pair, the coordinates of the feature points in the front image are affine transformed, the error from the coordinates of the feature points in the surveillance camera image is calculated, and the affine transformation parameters are determined so as to minimize the sum of the errors for each feature point pair. The affine transformation parameters thus obtained are output to the detection result transfer unit 61 as geometric transformation parameters.

[0028] The above-described method for calculating the geometric transformation parameters is merely an example, and any method for associating identical points between a pair of images is possible. For example, if the installation positions and angles of view of two cameras are given as meta-information, the transformation of the angles of view between the cameras can be analytically calculated to estimate identical points.

[0029] The object position estimation unit 20 (detection means) estimates the position of an object contained in an input image using a pre-trained object detection model. That is, during additional learning, the object position estimation unit 20 inputs a pair of a front image and a surveillance camera image and estimates the position of an object contained in each image. For the front image, a pre-trained object detection model generated by learning images with a similar angle of view in pre-learning is used, allowing for highly accurate estimation of the object position. On the other hand, for the surveillance camera image, images with a similar angle of view are not used in pre-learning, resulting in reduced object detection accuracy using the pre-trained object detection model. The object position estimation unit 20 outputs front image coordinates (first position) representing the position of the product in the front image, which are the estimation result of the object position for the front image, to the automatic answer assignment unit 60, and outputs surveillance camera image coordinates (second position) representing the position of the product in the surveillance camera image, which are the estimation result for the surveillance camera image, to the loss calculation unit 30.

[0030] The automatic answer assignment unit 60 (generation means) converts the front image coordinates, which are the estimation results of the object position estimation unit 20 for the front image, into coordinates in the surveillance camera image, using the geometric transformation parameters, which are the output of the geometric transformation estimation unit 50, and the rectangular coordinates, which indicate the position of the product in the front image, which are the output of the object position estimation unit 20. Specifically, the automatic answer assignment unit 60 converts the rectangular coordinates, which are the front image coordinates, which indicate the position of the object estimated by the object position estimation unit 20, according to the geometric transformation parameters estimated by the geometric transformation estimation unit 50, and finds converted coordinates (corresponding position) corresponding to the position of the object included in the surveillance camera image.

[0031] Specifically, the detection result transfer unit 61 converts the position of the object included in the front image estimated by the object position estimation unit 20 using the affine transformation parameters calculated by the geometric transformation parameter calculation unit 53, and calculates the corresponding position of the object on the surveillance camera image. In this embodiment, the detection result transfer unit 61 converts the coordinates of four points of a rectangle, which are front image coordinates indicating the position of the object included in the front image estimated by the object position estimation unit 20, using the affine transformation parameters calculated by the geometric transformation parameter calculation unit 53, and outputs the coordinate values ​​of the four points, which are the transformed coordinates obtained by the conversion, to the solution rectangle generation unit 62.

[0032] The solution rectangle generation unit 62 converts the transformed coordinates of the object position on the surveillance camera image calculated by the detection result transfer unit 61 into rectangular coordinates for further training an object detection model. For example, the solution rectangle generation unit 62 calculates the smallest rectangle that encloses four points that are the transformed coordinates representing the object position calculated by the detection result transfer unit 61, and outputs the coordinates of the four vertices of this smallest rectangle to the loss calculation unit 30 as a solution rectangle (position information).

[0033] FIG. 5 shows an example of a pair of front images P1 and surveillance camera images P2. The pair of front images P1 and surveillance camera images P2 are images of the same product shelves captured at the same time within a range where objects do not move, and therefore the same products are displayed in the same arrangement on the product shelves in both images. The rectangle surrounding the products shown in front image P1 is an example of front image coordinates representing the position of the products in the front image detected from front image P1, which is the output result of object position estimation unit 20. Surveillance camera image P2 is an image of a product shelf captured at a different angle of view from the front image. Note that surveillance camera image P2 does not show surveillance camera image coordinates representing the position of the products in the surveillance camera image detected from surveillance camera image P2, which is the output result of object position estimation unit 20.

[0034] Then, using the pair of front image and surveillance camera image described above, the geometric transformation estimation unit 50 estimates geometric transformation parameters so that the angles of view of both images match. Furthermore, the automatic answer assignment unit 60 uses the geometric transformation parameters estimated by the geometric transformation estimation unit 50 to convert front image coordinates representing the position of the product in the front image, which is the estimation result of the object position estimation unit 20 and shown in the front image P1, into coordinates on the surveillance camera image P2.

[0035] FIG. 6 shows how the automatic correct answer assignment unit 60 converts rectangular coordinates representing an object position. Reference symbol P11 in FIG. 6 indicates one of the products shown in the front image P1 and the product position rectangle whose position is estimated by the object position estimation unit 20. Reference symbol P12 in FIG. 6 indicates the dotted line rectangle resulting from the product position rectangle in the front image shown in reference symbol P11 being transformed using geometric transformation parameters in the detection result transfer unit 61 and transferred to the surveillance camera image P2. Since the dotted rectangle indicated by reference symbol P12 is not suitable as input to the loss calculation unit 30 (described later), the correct answer rectangle generation unit 62 converts it into the smallest rectangle containing the dotted rectangle, i.e., a solid-line rectangle as indicated by reference symbol 13. The coordinates of the solid-line rectangle indicated by reference symbol 13 are output to the loss calculation unit 30 as the correct answer rectangle (correct answer data).

[0036] The loss calculation unit 30 (learning means) calculates a loss using the surveillance camera image coordinates (second position) representing the position of the product in the surveillance camera image, which are the output of the object position estimation unit 20, and the correct rectangle (position information based on the corresponding position) output by the automatic correction annotation unit 60, and updates the parameters of the object detection model of the object position estimation unit 20 in the same manner as in pre-learning, and performs learning. Specifically, the error between the rectangular coordinates, which are the surveillance camera image coordinates that are the estimation result of the object detection model for the surveillance camera image, and the correct rectangular coordinates of the object position calculated by the automatic correction annotation unit 60 is calculated and used as the loss. The loss calculation unit 30 then updates the parameters of the object detection model to reduce the loss. The parameters of the object detection model are updated until the loss value converges to a predetermined value or less, and pre-learning of the object detection model ends when the loss value converges. The object detection model at the end of learning is obtained as the trained object detection model.

[0037] The trained object detection model obtained as described above is later used to detect objects from images to be inferred. Specifically, the object position estimation unit 20 inputs a surveillance camera image to be inferred, uses the trained object detection model to estimate rectangular coordinates that represent the position of an object included in the input image, and outputs the result.

[0038] In the above configuration, the object position estimation unit 20 is an example of a detection unit, the geometric deformation estimation unit 50 and the automatic solution assignment unit 60 are examples of a generation unit, and the loss calculation unit 30 is an example of a learning unit.

[0039] [Operation of Object Detection Device] Next, the operation of the object detection device 100 will be described. Figure 7 is a flowchart of the learning process for the object detection model, and particularly illustrates the operation of the additional learning described above. For this reason, it is assumed that the object detection device 100 has previously performed the above-described pre-learning and generated a basic object detection model. It is also assumed that the image DB 2 stores an image pair consisting of a front image and a surveillance camera image taken of the same product shelf at the same time within an area where no objects are moving.

[0040] First, the object detection device 100 inputs an image pair, a front image and a surveillance camera image, from the image DB 2 to the geometric transformation estimation unit 50 and the object position estimation unit 20 (step S11). The geometric transformation estimation unit 50 estimates geometric transformation parameters between the two input images and inputs them to the automatic correction assignment unit 60 (step S12). The object position estimation unit 20 estimates rectangular coordinates representing the positions of objects included in the two images and inputs the estimation results to the automatic correction assignment unit 60 and the loss calculation unit 30 (step S13). Specifically, the object position estimation unit 20 inputs the correct image coordinates, which are the estimation results for the front image, to the automatic correction assignment unit 60, and inputs the surveillance camera image coordinates, which are the estimation results for the surveillance camera image, to the loss calculation unit 30. Note that the process of estimating the object position in step S13 may be performed before the process of estimating the geometric transformation parameters in step S12.

[0041] Next, the automatic answer assignment unit 60 calculates rectangular coordinates of the object position included in the surveillance camera image using inputs from the geometric transformation estimation unit 50 and the object position estimation unit 20, and inputs the calculated rectangular coordinates to the loss calculation unit 30 (step S14). Specifically, the automatic answer assignment unit 60 converts rectangular coordinates, which are front image coordinates representing the object position estimated from the front image by the object position estimation unit 20, according to the geometric transformation parameters estimated by the geometric transformation estimation unit 50, to obtain transformed coordinates corresponding to the position on the surveillance camera image, and inputs the transformed coordinates to the loss calculation unit 30. Note that the process of generating transformed coordinates from the front image coordinates in step S14 may be performed before the process of estimating the object position from the surveillance camera image in step S13. In other words, the process of estimating the object position from the surveillance camera image in step S13 may be performed after step S14.

[0042] The loss calculation unit 30 calculates the loss using the rectangular coordinates input from the automatic correction assignment unit 60 and the object position estimation unit 20 (step S15). Specifically, the loss calculation unit 30 calculates the loss using the surveillance camera image coordinates representing the position of the product in the surveillance camera image, which are output from the object position estimation unit 20, and the correct rectangular coordinates, which are output from the automatic correction assignment unit 60. Then, the loss calculation unit 30 determines whether the loss has converged to a predetermined value or less (step S16). If the loss has not converged (step S16: No), the loss calculation unit 30 updates the parameters of the object detection model constituting the object position estimation unit 20 so as to reduce the loss (step S17). Then, the process returns to step S11. On the other hand, if the loss has converged (step S16: Yes), the process ends.

[0043] The object detection device 100 can then input the surveillance camera image to be inferred, estimate the coordinates representing the position of the object contained in the input image using the trained object detection model, and output the result.

[0044] As described above, the object detection model generation device of the first embodiment trains an object detection model using a pair of images with different angles of view, such as a frontal image and a surveillance camera image, thereby enabling accurate object detection within images, even for new images, such as surveillance camera images with a different angle of view from the frontal image. Since the object positions in the frontal image and the surveillance camera image are automatically annotated, it is possible to generate an object detection model that can accommodate images with new angles of view while keeping the cost of manual annotation low.

[0045] In the above embodiment, the case where the object to be detected is a product displayed on a product shelf has been described as an example, but the application of the present disclosure is not limited to product detection. For example, in a surveillance camera for people, abandoned object detection, product monitoring, etc., it is possible to apply the present disclosure to a site where learning images taken from multiple angles of view within a range in which the position of the object does not change can be obtained.

[0046] Second Embodiment Next, a second embodiment of the present disclosure will be described with reference to Fig. 8. Fig. 8 is a block diagram showing the configuration of a model generation device in the second embodiment. Note that this embodiment shows an outline of the configuration of the object detection device described in the above-mentioned embodiment.

[0047] The model generating device 200 in this embodiment is configured as a general information processing device, and as an example, is equipped with the same hardware configuration as the object detection device described in embodiment 1. In other words, the model generating device is equipped with components such as a communication unit, a processor, a memory, and a recording medium.

[0048] The model generating device 200 can be equipped with the detecting means 201, generating means 202, and learning means 203 shown in Figure 8 by having a processor retrieve and execute a program stored in a memory or storage medium. The program may be supplied to the processor via a communications network, or may be stored in a storage medium in advance and read out by a drive device and supplied to the processor. However, the detecting means 201, generating means 202, and learning means 203 described above may be constructed using dedicated electronic circuits for realizing these means.

[0049] The detection means 201 uses an object detection model to detect a first position, which is the position of an object in a first image, and a second position, which is the position of the object in a second image, which has a different angle of view from the first image. The first and second images are images of the same object on which the object is located, but have different angles of view. For example, the images are images of a store shelf on which the objects are displayed. For example, the first image is an image of the store shelf taken from the front, and the second image is a surveillance camera image of the store shelf taken by a surveillance camera installed on the ceiling, etc. The detection means 201 then detects the positions (first and second positions) of the products displayed on the store shelf from the front image and the surveillance camera image, respectively. Note that the object detection model at this point has been trained mainly using images taken with the angle of view of the first image, so the object detection accuracy from the first image is high, but the object detection accuracy from the second image is low.

[0050] The generating means 202 generates a corresponding position from a first position, which is a position in the second image corresponding to the first position, based on the difference in angle of view between the first image and the second image. For example, the generating means 202 estimates the difference in angle of view between the first image and the second image and generates deformation parameters for transforming the first image into the second image. The generating means 202 then generates a corresponding position by transforming the position of an object in the first image, for example, the position of the object in the front image (first position), using the generated deformation parameters. As a result, a corresponding position on the second image, which has a different angle of view, is generated from the first position detected from the first image with high accuracy.

[0051] The learning means 203 learns an object detection model based on the second position and the corresponding position. For example, the learning means 203 updates and learns the parameters of the object detection model using the corresponding position as correct data for the second position. As a result, the position of an object detected using the object detection model from the second image, for example, a surveillance camera image, is learned to approach the corresponding position.

[0052] With the above-described configuration, the present disclosure can use the generated object detection model to accurately detect the position of an object even in a second image that has a different angle of view from the first image.

[0053] Although the present disclosure has been described above with reference to the above-described embodiments, the present disclosure is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present invention. Furthermore, at least one or more functions of the above-described detection means, generation means, and learning means may be executed by an information processing device installed and connected anywhere on a network, i.e., may be executed by so-called cloud computing.

[0054] <Supplementary Notes> Some or all of the above embodiments can also be described as in the following supplementary notes. Below, an outline of the configurations of a model generation device, a model generation method, and a program according to the present invention will be described. However, the present invention is not limited to the following configurations. (Supplementary Note 1) A model generation device comprising: a detection means that uses an object detection model to detect a first position that is the position of an object in a first image and a second position that is the position of the object in a second image that has a different angle of view from the first image; a generation means that generates, from the first position, a corresponding position that is a position in the second image that corresponds to the first position, based on a difference in angle of view between the first image and the second image; and a learning means that learns the object detection model based on the second position and the corresponding position. (Supplementary Note 2) A model generation device according to Supplementary Note 1, wherein the generation means calculates deformation parameters for transforming the first image into the second image based on the first image and the second image, and generates the corresponding position from the first position using the deformation parameters. (Supplementary Note 3) The model generation device according to Supplementary Note 2, wherein the generation means extracts first feature points in the first image and second feature points in the second image that correspond to the first feature points, and calculates the deformation parameters based on the first feature points and the second feature points. (Supplementary Note 4) The model generation device according to Supplementary Note 2, wherein the generation means deforms a rectangular area corresponding to the first position using the deformation parameters to generate the corresponding position. (Supplementary Note 5) The model generation device according to Supplementary Note 1, wherein the learning means learns the object detection model so as to reduce an error between the second position, which is a position of an object in the second image detected using the object detection model, and position information based on the corresponding position. (Supplementary Note 6) The model generation device according to Supplementary Note 5, wherein the learning means learns the object detection model so as to reduce an error between the second position, which is the position of the object in the second image detected using the object detection model, and position information consisting of a rectangle including an area of ​​the corresponding position.(Supplementary Note 7) A model generation method that uses an object detection model to detect a first position that is the position of an object in a first image and a second position that is the position of the object in a second image that has a different angle of view from the first image, generates a corresponding position from the first position that is a position in the second image that corresponds to the first position based on a difference in angle of view between the first image and the second image before and after the detection of the second position, and trains the object detection model based on the second position and the corresponding position. (Supplementary Note 8) The model generation method according to Supplementary Note 7, further comprising: calculating, based on the first image and the second image, deformation parameters for transforming the first image into the second image, and generating the corresponding position from the first position using the deformation parameters. (Supplementary Note 9) The model generation method according to Supplementary Note 7, further comprising: training the object detection model so as to reduce an error between the second position that is the position of the object in the second image detected using the object detection model and position information based on the corresponding position. (Supplementary Note 10) A computer-readable storage medium storing a program for causing a computer to execute the following processes: using an object detection model, detecting a first position that is the position of an object in a first image and a second position that is the position of the object in a second image that has a different angle of view from the first image; generating a corresponding position that is a position in the second image that corresponds to the first position from the first position based on the difference in angle of view between the first image and the second image; and learning the object detection model based on the second position and the corresponding position.

[0055] 2 Image database 3 Product shelf 4 Surveillance camera 5 Camera of mobile terminal 6 Terminal device 20 Object position estimation unit 30 Loss calculation unit 50 Geometric transformation calculation unit 51 Feature point extraction unit 52 Feature point matching degree calculation unit 53 Geometric transformation parameter estimation unit 60 Automatic correction assignment unit 61 Detection result transfer unit 62 Correct rectangle generation unit 100 Object detection device 101 Communication unit 102 Processor 103 Memory 104 Recording medium

Claims

1. A detection means for detecting a first position, which is a position of an object in a first image, and a second position, which is a position of an object in a second image having a different angle of view from the first image, using an object detection model; a generating means for generating, from the first position, a corresponding position in the second image corresponding to the first position, based on a difference in angle of view between the first image and the second image; a learning means for learning the object detection model based on the second position and the corresponding position; A model generating device comprising:

2. 2. The model generating device according to claim 1, the generating means calculates a transformation parameter for transforming the first image into the second image based on the first image and the second image, and generates the corresponding position from the first position using the transformation parameter; Model generation device.

3. 3. The model generating device according to claim 2, the generating means extracts a first feature point in the first image and a second feature point in the second image corresponding to the first feature point, and calculates the deformation parameters based on the first feature point and the second feature point; Model generation device.

4. 3. The model generating device according to claim 2, the generating means transforms a rectangular area corresponding to the first position using the transformation parameters to generate the corresponding position; Model generation device.

5. 2. The model generating device according to claim 1, the learning means learns the object detection model so as to reduce an error between the second position, which is a position of an object in the second image detected using the object detection model, and position information based on the corresponding position; Model generation device.

6. 6. A model generating device according to claim 5, the learning means learns the object detection model so as to reduce an error between the second position, which is a position of an object in the second image detected using the object detection model, and position information consisting of a rectangle including an area of ​​the corresponding position; Model generation device.

7. Using an object detection model, a first position that is a position of an object in a first image and a second position that is a position of an object in a second image having a different angle of view from the first image are detected; before or after the detection of the second position, a corresponding position is generated from the first position, which is a position in the second image corresponding to the first position, based on a difference in angle of view between the first image and the second image; training the object detection model based on the second position and the corresponding position; Model generation method.

8. 8. A model generation method according to claim 7, comprising the steps of: calculating a transformation parameter for transforming the first image into the second image based on the first image and the second image, and generating the corresponding position from the first position using the transformation parameter; Model generation method.

9. 8. A model generation method according to claim 7, comprising the steps of: learning the object detection model so as to reduce an error between the second position, which is a position of an object in the second image detected using the object detection model, and position information based on the corresponding position; Model generation method.

10. Using an object detection model, a first position that is a position of an object in a first image and a second position that is a position of an object in a second image having a different angle of view from the first image are detected; generating a corresponding position, which is a position in the second image corresponding to the first position from the first position, based on a difference in angle of view between the first image and the second image; training the object detection model based on the second position and the corresponding position; A program that causes a computer to execute a process.