IMAGE JUDGMENT DEVICE, IMAGE JUDGMENT METHOD, AND PROGRAM
The image determination device uses R-CNN and subsequent convolutional models to accurately assess image identity by extracting and comparing object data, addressing the challenges of environmental variations in image comparison.
Patent Information
- Application Number
- JP2023206941
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2023-12-07
- Publication Date
- 2024-11-21
- Estimated Expiration
- 2040-09-08
AI Technical Summary
Existing methods struggle to accurately determine the substantial identity of images across varying execution environments due to differences in screen sizes, aspect ratios, and object appearances, and machine learning approaches face significant learning challenges.
An image determination device utilizing a first machine learning model, such as R-CNN, to extract object data from images, followed by a second model with convolution and pooling layers to reduce dimensions and a fully connected layer for identity determination, enabling accurate comparison of image attributes and arrangements.
The solution achieves an accuracy of approximately 86% in determining the substantial identity of images, effectively handling variations in screen sizes and object placements.
Smart Images

Figure 0007588203000001 
Figure 0007588203000002 
Figure 0007588203000003
Abstract
Description
[Technical field]
[0001] The present invention relates to an image judgment device, an image judgment method, and a program, and more particularly to a technique for judging the substantial identity of a plurality of images. [Background technology]
[0002] In the field of computer technology, there is a great need to determine whether a plurality of images, each of which has an object arranged therein, are substantially identical. For example, a screen image of a computer application is manually designed on the assumption that it will be displayed in one selected execution environment, such as a specific smartphone. Such a screen image has objects arranged therein, such as buttons, images, and input forms. When the design of a screen image for one execution environment is completed, the screen image with a substantially identical design is also ported to other execution environments. This makes it possible to provide a substantially identical screen image in various execution environments. For example, a screen image designed for a specific smartphone of a specific manufacturer is ported to other smartphones of the same manufacturer or smartphones of different manufacturers by manual or automatic conversion using computer software. Conventionally, the substantial identity of the plurality of screen images obtained in this manner has often been confirmed by visual inspection. However, as the number of execution environments for computer applications increases, the need to automate such confirmation work is increasing. Summary of the Invention [Problem to be solved by the invention]
[0003] Screen sizes, aspect ratios, and resolutions vary depending on the execution environment of a computer application. In addition, objects such as buttons provided by an execution environment such as an operating system and included in each screen image have somewhat different appearances. For this reason, it is difficult to confirm the substantial identity of multiple screen images even if they are compared pixel by pixel. In addition, it is possible to input the screen images into a machine learning model to determine the substantial identity of the images, but there is a concern that the amount of learning required would be enormous.
[0004] In addition to screen images of computer applications, there are many needs to determine the substantial identity of multiple images containing objects, such as page images of e-books viewed in various environments and web content images viewed in various environments. The present invention has been made in consideration of the above-mentioned problems, and its object is to provide an image judgment device, an image judgment method, and a program that can easily and accurately judge the substantial identity of multiple images in which objects are placed. [Means for solving the problem]
[0005] In order to solve the above problem, an image judgment device according to one aspect of the present invention includes an object data acquisition means for acquiring first object data indicating attributes and arrangement of objects in a first image from a first image using a first machine learning model that receives an image and outputs object data indicating attributes and arrangement of objects in the image, and acquiring second object data indicating attributes and arrangement of objects in the second image from a second image, and a judgment means for inputting the first object data and the second object data, and for judging substantial identity of the first image and the second image using a second machine learning model that outputs substantial identity of the first image and the second image.
[0006] Here, the first machine learning model may include R-CNN.
[0007] Additionally, the first machine learning model may be trained using training images generated by superimposing some or all of a plurality of objects onto a given base image.
[0008] Additionally, the second machine learning model may include a fully connected layer.
[0009] The second machine learning model may include a convolutional layer and a pooling layer upstream of the fully connected layer that reduce the dimensionality of input data based on the first object data and the second object data.
[0010] In addition, the second machine learning model may be trained using first learning object data indicating the attributes and arrangement of objects in a first training image and second learning object data indicating the attributes and arrangement of objects in the second training image, which are obtained from a first training image and a second training image generated by superimposing a given object on each of an identical or similar first and second base images according to a predetermined arrangement rule.
[0011] Moreover, an image judgment method according to another aspect of the present invention includes the steps of: acquiring, from a first image, first object data indicating attributes and arrangement of objects in the first image, using a first machine learning model; acquiring, from a second image, second object data indicating attributes and arrangement of objects in the second image, using the first machine learning model; and determining substantial identity between the first image and the second image from the first object data and the second object data, using a second machine learning model.
[0012] Here, the first machine learning model may include R-CNN.
[0013] The method may further include a step of performing learning of the first machine learning model using training images generated by superimposing some or all of a plurality of objects on a given base image.
[0014] Additionally, the second machine learning model may include a fully connected layer.
[0015] The second machine learning model may also include a convolutional layer and a pooling layer upstream of the fully connected layer that reduce the number of dimensions of input data based on the first object data and the second object data.
[0016] The above method may further include a step of generating a first training image and a second training image by superimposing a given object on each of identical or similar first and second base images in accordance with a predetermined arrangement rule, a step of inputting each of the first training image and the second training image into the first machine learning model to obtain first learning object data indicating attributes and arrangement of objects in the first training image, and second learning object data indicating attributes and arrangement of objects in the second training image, and a step of performing learning of the second machine learning model using the first learning object data and the second learning object data.
[0017] Furthermore, a program according to yet another aspect of the present invention is a program for causing a computer to execute the steps of: acquiring, from a first image, first object data indicating attributes and arrangement of objects in the first image, using a first machine learning model; acquiring, from a second image, second object data indicating attributes and arrangement of objects in the second image, using the first machine learning model; and determining substantial identity between the first image and the second image from the first object data and the second object data, using a second machine learning model. This program may be stored in a computer-readable information storage medium such as a magneto-optical disk or a semiconductor memory. [Brief description of the drawings]
[0018] [Figure 1]FIG. 1 is a block diagram of an image assessment device according to an embodiment of the present invention. [Diagram 2] FIG. 13 is a diagram showing an example of identity determination between two screen images. [Diagram 3] FIG. 13 is a block diagram of an image assessment device according to another embodiment. [Figure 4] FIG. 1 is a diagram illustrating an example of the configuration of a CNN. [Diagram 5] FIG. 13 is a diagram illustrating another example of the configuration of a CNN. [Figure 6] FIG. 1 is a block diagram of a learning device used for training R-CNN. [Figure 7] FIG. 2 is a diagram illustrating an example of a base image. [Figure 8] FIG. 13 is a diagram showing an example of an object image. [Figure 9] FIG. 13 is a diagram illustrating an example of an object attribute table. [Figure 10] FIG. 13 is a diagram showing an example of a training image. [Figure 11] FIG. 11 is a diagram showing correct answer data corresponding to the training image shown in FIG. [Figure 12] FIG. 1 is a flow diagram showing the learning process of R-CNN. [Figure 13] FIG. 1 is a block diagram of a learning device used for CNN training. [Figure 14] FIG. 1 is a diagram showing training data relating to positive examples and negative examples used in learning a CNN. [Figure 15] FIG. 1 is a flow diagram showing a learning process of a CNN. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0019] An embodiment of the present invention will be described in detail with reference to the drawings. In the following description, the same parts are designated by the same reference numerals in each drawing, and the description will be omitted as appropriate.
[0020] Fig. 1 is a functional block diagram of an image judgment device according to an embodiment of the present invention. Each functional block shown in the figure is realized by executing an image judgment program according to an embodiment of the present invention on a computer mainly composed of a CPU and a memory. The image judgment device 10 shown in the figure judges the substantial identity of two screen images.
[0021] As described above, objects such as buttons are arranged on each screen image. The image assessment device 10 assesses the substantial identity of the two screen images based on the number of objects included in the two screen images, the attributes of each object, and the arrangement of each object in the screen images. Here, the attributes of an object are, for example, the type and color information of the object. Examples of object types include buttons, logo images, trademark images, and input forms. The color information of an object may be information on one or more representative colors of the object, information on an average color, or the like.
[0022] Here, two images are "substantially identical" when the number of object images contained in each image, their attributes, and their arrangement match predefined positive example rules. Also, two images are "not substantially identical" when the number of object images contained in each image, their attributes, and their arrangement match predefined negative example rules.
[0023] According to the image assessment device 10 shown in Fig. 1, when object A is placed in the upper left of a screen image as shown in Fig. 2(a) and object B is placed in the lower right, a screen image in which only object A is placed in the same position and object B is missing as shown in Fig. 2(b) is judged to be substantially not identical to the screen image shown in Fig. 2(a). In other words, a screen image with a mismatch in the number of objects is judged to be substantially not identical.
[0024] In addition, as shown in Fig. 1(c), a screen image in which object A is placed in the same position but object B is placed with a horizontal shift is also determined to be substantially not identical to the screen image shown in Fig. 1(a). In other words, a screen image in which corresponding objects are placed with a horizontal shift is also determined to be substantially not identical to the screen image shown in Fig. 1(a).
[0025] On the other hand, a screen image in which the size of object B is different, as shown in Fig. 1(d), or a screen image in which the size of object A is different, as shown in Fig. 1(e), are judged to be substantially identical to the screen image shown in Fig. 1(a). In other words, when comparing two screen images, if the number, attributes, and arrangement of objects are the same, the two screen images are judged to be substantially identical even if the sizes of the objects are different.
[0026] In order to realize such a judgment of substantial identity, as shown in FIG. 1, the image judgment device 10 includes an R-CNN (Regions with Convolutional Neural Networks) 12, which is a first machine learning model, and a CNN (Convolutional Neural Networks) 14, which is a second machine learning model. When a screen image A of two screen images to be compared is input to the R-CNN 12, object data A indicating the attributes and arrangement of objects included in the screen image A is output. When a screen image B of the two screen images is input, object data B indicating the attributes and arrangement of objects included in the screen image B is output. These object data A and object data B are input to the CNN 14. The CNN 14 outputs whether or not the screen images A and B are substantially identical based on the object data A and object data B.
[0027] In this embodiment, the screen image A and the screen image B are sequentially input to the R-CNN 12, and the object data A and the object data B are sequentially obtained. However, as shown in FIG. 3, two R-CNNs 12 may be arranged in parallel in front of the CNN 14, and object data B may be generated from the screen image B in parallel with the generation of object data A from the screen image A.
[0028] In FIG. 1, R-CNN12 is a machine learning model for object detection, and may be configured by a known machine learning model for object detection, such as original R-CNN, Fast R-CNN, Faster R-CNN, or MASK R-CNN. In this embodiment, Faster R-CNN is used as an example of R-CNN12. When an image is input to R-CNN12, object data indicating the arrangement in the input image and the attributes of each object included in the image is output. The arrangement of each object in the input image is specified by, for example, an anchor box in the object data.
[0029] As shown in FIG. 4, the CNN 14 includes a data integration unit 15, a multi-stage dimension reduction unit 16, a one-dimensionalization unit 17, and a multi-stage fully connected layer 18, each of which is composed of a set of a convolution layer and a pooling layer. The data integration unit 15 concatenates the object data A and the object data B output from the R-CNN 12 to generate one input data. For example, the object data A and the object data B may be four-dimensional tensor data consisting of one dimension indicating the ordinal number of the object (e.g., 300 pieces of data), two dimensions indicating the size and arrangement of the object (e.g., each dimension may include four pieces of data), and one dimension indicating the attribute of the object (e.g., 1024 pieces of data). The data integration unit 15 may, for example, expand the dimension of the ordinal number of the object to twice the size, thereby combining the object data A and the object B to obtain the input data. In this case, the ordinals 1 to 300 of the object are assigned to the objects included in the screen image A, and the ordinals 301 to 600 of the object are assigned to the objects included in the screen image B.
[0030] The input data generated by the data integration unit 15 is subjected to dimensionality reduction by multiple stages of dimensionality reduction units 16, and two-dimensional intermediate data is output from the final-stage dimensionality reduction unit 16. The one-dimensionalization unit 17 converts the intermediate data into one dimension, and inputs the one-dimensional intermediate data to the first-stage fully connected layer 18. The final-stage fully connected layer 18 outputs a one-dimensional (which may include two pieces of data) identity determination result from the one-dimensional intermediate data. The identity determination result includes data indicating the degree of identity between screen image A and screen image B, and data indicating the degree to which such identity does not exist.
[0031] According to the CNN 14 shown in FIG. 4, it becomes possible to determine whether or not the screen image A and the screen image B are substantially identical, based on the attributes and arrangement of objects contained in the screen image A and the screen image B, respectively.
[0032] In FIG. 4, object data A and object data B are combined to obtain input data, and the dimensionality of the input data is reduced. However, dimensionality reduction may be performed on each of object data A and object data B. That is, as in the CNN 14 shown in FIG. 5, a multi-stage dimensionality reduction unit 16a including a convolution layer and a pooling layer may reduce the dimensionality of object data A, and a dimensionality reduction unit 16b having the same function as the dimensionality reduction unit 16a may reduce the dimensionality of object data B in parallel with the dimensionality reduction of object data A. The outputs of the dimensionality reduction units 16a and 16b are two-dimensional intermediate data, and the data integration unit 19 converts each intermediate data into one dimension and concatenates them to output one one-dimensional intermediate data. The intermediate data output from the data integration unit 19 is input to a multi-stage fully connected layer 18, and an identity determination result is output from the final stage.
[0033] Using the CNN 14 shown in FIG. 5 also makes it possible to determine whether or not the screen images A and B are substantially identical, based on the attributes and arrangement of objects contained in the screen images A and B, respectively.
[0034] Here, the learning of R-CNN12 and CNN14 will be explained.
[0035] 6 is a diagram showing the configuration of a learning device 20 that performs learning of R-CNN12. Each functional block shown in the figure is realized by executing a program according to an embodiment of the present invention on a computer mainly configured with a CPU and a memory. As shown in the figure, the learning device 20 includes a training unit 21 that performs learning of R-CNN12 by updating the internal parameters of R-CNN12, a training data generation unit 22 that generates training data used for learning of R-CNN12, a base image storage unit 24 that stores a base image used when generating the training data, and an object image storage unit 26 that stores an object image used when generating the training data.
[0036] FIG. 7 shows an example of a base image stored in the base image storage unit 24. A base image is raster-format image data that forms the basis of a screen image, and FIG. 7(a) shows a base image in which text is displayed across the entire surface. FIG. 7(b) shows a plain white base image. FIG. 7(c) shows a base image consisting of a plain colored page upper portion and a document displayed below. FIG. 7(d) shows a base image that is divided into multiple blocks, with a document displayed in each block. The base image storage unit 24 stores a large number of such raster-format base images.
[0037] FIG. 8 shows an example of an object image stored in the object image storage unit 26. An object image is raster-format image data that is the basis of a screen image, and shows the appearance of an object such as a button or an input form that is arranged in the screen image. FIG. 8(a) shows a logo image, which is a type of object image. The object image shown in FIG. 8 is assigned the object number "001". FIG. 8(b) shows an input form image, which is a type of object image. The object image shown in FIG. 8 is assigned the object number "002". FIG. 8(c) shows a button image, which is a type of object image. The object image shown in FIG. 8 is assigned the object number "003". FIG. 8(d) shows a trademark image, which is a type of object image. The object image shown in FIG. 8 is assigned the object number "004". The object image storage unit 26 stores a large number of such raster-format object images.
[0038] In addition to a large number of object images, the object image storage unit 26 stores an object attribute table shown in Fig. 9. The object attribute table stores attributes of each object image stored in the object image storage unit 26. Specifically, the object attribute table stores, in association with each object number, the type and color information of the object image related to the number. For example, the object attribute table stores, in association with the object number "001" of the object image shown in Fig. 8(a), the type of the object image "logo" and the color information of the object image "red and white".
[0039] The training data generating unit 22 generates a large amount of training data based on the data stored in the base image storage unit 24 and the object image storage unit 26. The training data includes training images and correct answer data.
[0040] Fig. 10 shows an example of a training image generated by the training data generating unit 22. The training image shown in Fig. 10(a) is generated by superimposing the object image shown in Fig. 8(a), the object image shown in Fig. 8(d), and the object image shown in Fig. 8(b) on the base image shown in Fig. 7(b). The training image shown in Fig. 10(b) is generated by superimposing the object image shown in Fig. 8(b) and the object image shown in Fig. 8(c) on the base image shown in Fig. 7(a).
[0041] The base image used when generating the training image is selected randomly by the training data generating unit 22 from among the many base images stored in the base image storage unit 24. In addition, the object image to be superimposed on the base image is also selected randomly by the training data generating unit 22 from among the many base images stored in the object image storage unit 24. Furthermore, the arrangement (position and size) of each object image is also determined randomly by the training data generating unit 22.
[0042] FIG. 11 shows an example of correct answer data generated by the training data generating unit 22. FIG. 11(a) shows correct answer data for the training image shown in FIG. 10(a). This correct answer data shows that the training image includes three object images, the first object image is arranged as "(XA1,YA1)-(XB1,YB1)" and has the attribute "logo, red / white", the second object image is arranged as "(XA2,YA2)-(XB2,YB2)" and has the attribute "mark, yellow", and the third object image is arranged as "(XA3,YA3)-(XB3,YB3)" and has the attribute "form, white / black". That is, the correct answer data shows the number of object images included in the training image, and the arrangement and attribute of each object image. Similarly, FIG. 11(b) shows correct answer data for the training image shown in FIG. 10(b).
[0043] When generating training images, the training data generating unit 22 randomly selects an object image to be superimposed on the base image and also randomly determines its arrangement. The training data generating unit 22 reads out the attributes of the selected object image from the object attribute table shown in Fig. 9 and includes them in the correct answer data shown in Fig. 11, and also includes the determined arrangement of the object image in the correct answer data shown in Fig. 11, thereby generating correct answer data corresponding to the training image.
[0044] The training unit 21 executes the learning process of the R-CNN 12 using the training data generated by the training data generation unit 22. Specifically, the training images included in the training data are sequentially input to the R-CNN 12, and object data is output. The difference between this output and the ground truth data included in the training data is calculated, and the internal parameters of the R-CNN 12 are updated so that this difference becomes smaller.
[0045] 12 is an operation flow diagram of the learning device 20. In the learning device 20, the training data generation unit 22 first randomly selects object images stored in the object image storage unit 26 (S101). Here, the number of object images to be selected is also determined randomly. Next, the training data generation unit 22 randomly selects one base image from the base image storage unit 24 (S102).
[0046] The training data generating unit 22 generates training images by arranging the selected object images on the selected base image (S103). At this time, the training data generating unit 22 randomly determines the arrangement and size of each object image.
[0047] The training data generating unit 22 further generates correct answer data as exemplified in FIG. 11 according to the processing contents in S101 to S103, and includes the correct answer data together with the training image generated in S103 in the training data (S104).
[0048] The processes of S101 to S104 are repeated until a predetermined number of training data are generated (S105), and then the training unit 21 executes a learning process of the R-CNN 12 using the generated training data (S106).
[0049] Next, FIG. 13 is a diagram showing the configuration of a learning device 30 that performs learning of the CNN 14. Each functional block shown in the figure is realized by executing a program according to an embodiment of the present invention on a computer mainly configured with a CPU and a memory. As shown in the figure, the learning device 30 includes a training unit 31 that performs learning of the CNN 14 by updating the internal parameters of the CNN 14, a training data generation unit 32 that generates training data used for learning the CNN 14, a base image storage unit 24, and an object image storage unit 26. The training data generation unit 32 generates training data related to positive examples and negative examples using the base images stored in the base image storage unit 24 and the object images stored in the object image storage unit 26. Each training data includes positive examples, i.e., pairs of training images that are substantially identical, or negative examples, i.e., pairs of training images that are not substantially identical.
[0050] Fig. 14 is a diagram showing training images relating to positive examples and negative examples used for learning of the CNN 14. Fig. 14(a) shows one example of a pair of training images. The one training image is generated by placing a plurality of randomly selected object images at random positions and with random sizes on a randomly selected base image. In Fig. 14(a), object images A and B are placed on a base image in which a document is generally displayed.
[0051] FIG. 1B shows the other training image generated by applying the first negative example rule. In the first negative example rule, a part of the object image included in one training image is removed. For example, unlike the training image in FIG. 1A, the training image in FIG. 1B does not include object image B.
[0052] FIG. 1C shows the other training image generated by applying the second negative example rule. In the second negative example rule, a part of the object image included in one training image is shifted horizontally. For example, in the training image in FIG. 1C, object image B is shifted to the left compared to the training image in FIG. 1A.
[0053] Fig. 1D shows the other training image generated by applying the positive example rule. In the positive example rule, a part of the object image included in one training image is enlarged or reduced. For example, in the training image in Fig. 1D, object image A is enlarged compared to the training image in Fig. 1A.
[0054] The training unit 31 executes the learning process of the CNN 14 using the training data thus generated. Specifically, two training images included in the training data are input in sequence to the R-CNN 12 to obtain two object data. Then, the object data is input to the CNN 14. The training unit 31 obtains the identity determination result output from the CNN 14, and updates the internal parameters of the CNN 14 so that the identity determination result is correct. That is, when a training image related to a positive example is input to the R-CNN 12, the internal parameters are updated so that the identity determination result output from the CNN 14 indicates that the images are substantially identical. Conversely, when a training image related to a negative example is input to the R-CNN 12, the internal parameters are updated so that the identity determination result output from the CNN 14 indicates that the images are not substantially identical.
[0055] 15 is an operation flow diagram of the learning device 30. As shown in the figure, in the learning device 30, the training data generation unit 32 first randomly selects an object image from the object image storage unit 26 (S201). The training data generation unit 32 then randomly selects a base image from the base image storage unit 24 (S202). The training data generation unit 32 then superimposes each object image on the selected base image to generate a training image A (S203). At this time, the position and size of each object image are randomly determined.
[0056] Next, the training data generating unit 32 applies the positive example rule or the negative example rule to superimpose all or a part of the selected object image on the selected base image to generate training image B (S204). For example, when the first negative example rule is applied, training image B is generated without superimposing a part of the selected object image on the selected base image. When the second negative example rule is applied, training image B is generated by moving a part of the selected object image to the right or left and then superimposing it on the selected base image. When the positive example rule is applied, training image B is generated by enlarging or reducing a part of the selected object image and then superimposing it on the selected base image.
[0057] Thereafter, the training data generating unit 32 generates and stores training data including the generated pairs of training images A and B and a distinction between whether they are positive examples or negative examples (S205).
[0058] The processes of S201 to S205 are repeated until a predetermined number of training data are generated (S206), and then the training unit 31 executes the learning process of the CNN 14 using the generated training data (S207).
[0059] According to the image determination device 10 described above, object data indicating the attributes and positions of object images contained in the two screen images to be compared is obtained from each of the two screen images to be compared. Then, the presence or absence of substantial identity between the two screen images is determined based on the two object data. In order to determine the presence or absence of substantial identity, the CNN 14 is trained in advance by a large number of training image pairs generated according to the positive example rule and the negative example rule. According to this embodiment, the substantial identity between the two screen images can be suitably determined.
[0060] Specifically, the inventors used 500 base images and 33 types of object images to generate 5,000 training data for R-CNN12 and 8,000 training data for CNN14, and conducted training of R-CNN12 and CNN14. As a result, the accuracy of determining the substantial identity of screen images was approximately 86%, which was found to be sufficient for practical use.
[0061] The scope of the present invention is not limited to the above embodiment, but includes various modifications. For example, the present invention can be applied not only to screen images, but also to various images such as page images of electronic books and web content images. [Explanation of symbols]
[0062] 10 image judgment device, 12 R-CNN, 14 CNN, 15,19 data integration unit, 16,16a convolution layer and pooling layer, 17 one-dimensionalization unit, 18 fully connected layer, 20,30 learning device, 21,31 training unit, 22,32 training data generation unit, 24 base image storage unit, 26 object image storage unit.
Claims
1. an object data acquisition means for acquiring, from a first image, first object data which is the object data relating to an object in the first image, and acquiring, from a second image, second object data which is the object data relating to an object in the second image, using a first machine learning model which receives an image and outputs object data relating to an object in the image; a determination means for determining whether the first image and the second image are substantially the same by using a second machine learning model to which the first object data and the second object data are input and which outputs the substantial identity of the first image and the second image; Including, the first image being a screen image of a computer application displayed in a first execution environment; the second image is a screen image of a computer application displayed in a second execution environment different from the first execution environment; The image assessment device according to the present invention is characterized in that
2. the object data includes data indicating an arrangement of objects in the image; The image assessment device according to claim 1 .
3. the second machine learning model includes a fully connected layer; the second machine learning model is trained by first learning object data, which is the object data related to an object in a first training image, and second learning object data, which is the object data related to an object in a second training image, obtained from a first training image and a second training image generated by superimposing a given object on each of the same or similar first and second base images according to a predetermined arrangement rule; the first training image is an image generated by superimposing the given object at a given position and with a given size on the first base image; the second training image is an image generated without superimposing the given object on the second base image, the second machine learning model is trained assuming that the first training image and the second training image are not substantially identical; The image assessment device according to claim 2 .
4. the second machine learning model includes a fully connected layer; the second machine learning model is trained by first learning object data, which is the object data related to an object in a first training image, and second learning object data, which is the object data related to an object in a second training image, obtained from a first training image and a second training image generated by superimposing a given object on each of the same or similar first and second base images according to a predetermined arrangement rule; the first training image is an image generated by superimposing the given object at a given position and with a given size on the first base image; the second training image is an image generated by superimposing the given object on the second base image at a position different from the given position, the second machine learning model is trained assuming that the first training image and the second training image are not substantially identical; The image assessment device according to claim 2 .
5. the second machine learning model includes a fully connected layer; the second machine learning model is trained by first learning object data, which is the object data related to an object in a first training image, and second learning object data, which is the object data related to an object in a second training image, obtained from a first training image and a second training image generated by superimposing a given object on each of the same or similar first and second base images according to a predetermined arrangement rule; the first training image is an image generated by superimposing the given object at a given position and with a given size on the first base image; the second training image is an image generated by superimposing the given object on the second base image at a size different from the given size, the second machine learning model is trained assuming that the first training image and the second training image are substantially identical; The image assessment device according to claim 2 .
6. the object data includes data indicative of attributes of an object in the image; 6. The image assessment device according to claim 1,
7. A step of acquiring first object data from a first image, the first object data being object data related to an object in the first image, using a first machine learning model that receives an image and outputs object data related to an object in the first image; obtaining second object data from a second image using the first machine learning model, the second object data being the object data related to an object in the second image; determining substantial identity of the first image and the second image from the first object data and the second object data using a second machine learning model; Including, the first image being a screen image of a computer application displayed in a first execution environment; the second image is a screen image of a computer application displayed in a second execution environment different from the first execution environment; An image assessment method comprising:
8. A step of acquiring first object data from a first image, the first object data being object data related to an object in the first image, using a first machine learning model that receives an image and outputs object data related to an object in the first image; obtaining second object data from a second image using the first machine learning model, the second object data being the object data related to an object in the second image; determining substantial identity of the first image and the second image from the first object data and the second object data using a second machine learning model; on the computer, the first image being a screen image of a computer application displayed in a first execution environment; the second image is a screen image of a computer application displayed in a second execution environment different from the first execution environment; program.
Citation Information
Patent Citations
Comparison program, information processor, and comparison method
JP2016218977A
Analysis device, analysis system, and analysis program
JP2019219766A
Automated code generation
WO2019156706A1
