Image recognition system and image recognition program
By using data size reduction and multi-stage matching processes, the system achieves rapid and precise image recognition, addressing the inefficiencies of conventional systems in dynamic environments.
Patent Information
- Application Number
- JP2024008671
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2024-01-24
- Publication Date
- 2024-11-15
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Conventional image recognition systems are slow and inefficient in detecting object images, especially in applications with rapidly changing screens, making real-time image recognition challenging.
The system employs data size reduction techniques such as color reduction and downscaling, followed by first and second-stage matching rate calculations and determinations to identify and extract image components quickly and accurately.
This approach enables high-speed and high-precision image detection, allowing for faster and more accurate identification of objects within changing images, particularly in game development environments.
Smart Images

Figure 2024160929000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to an image recognition system capable of detecting a predetermined image from images to be searched. and image recognition programs. [Background technology]
[0002] Conventionally, various image recognition techniques for extracting other images that are the same as or related to one image have been used in various fields. As an example of such image recognition technology, Patent Document 1 describes The mobile device takes a specified image and performs a specified process on the taken image to enable high-speed communication. The remote server then stores the image in its own server. An image corresponding to the query image is extracted from the stored image data, and the image is Extracting information content associated with the image and displaying the extracted information content on a mobile device. It is described that the information is transmitted to the mobile device.
[0003] As another example of image recognition technology, Image recognition technology that detects specific image elements (object images) from among image elements is one example. This type of image recognition technology is used to detect the movement of the characters during a series of gameplay screens during game development. It is used for various purposes, such as debugging to detect whether there is a bug and fix the detected bug. It is being used.
[0004] Conventionally, in image recognition technology used for such debugging work, for example, A match rate is set to indicate how closely the object image matches the components contained in it. The algorithm is used to calculate the matching rate, and the components with a matching rate exceeding a predetermined threshold are classified as objects. The object image is detected by determining that the image is the same as the target image. was.
[0005] When using this type of image recognition technology for debugging during game development, A series of game screens that change from moment to moment as the game progresses are searched for each configuration in the target images. The matching rate between the element and the object image is calculated and the identity is judged. In order to calculate the successive match rate and determine the identity of the changing game screen, high-speed calculation processing is required. Conventionally, image recognition systems using GPUs (Graphics Processing Units) have been used. This type of high-speed processing was made possible by the system. [Prior art documents] [Patent documents]
[0006] [Patent Document 1] Special Publication No. 2012-524343 Summary of the Invention [Problem to be solved by the invention]
[0007] However, conventional image recognition systems take a long time to detect object images. For example, while using an application with fast screen transitions, It is difficult to perform image recognition processing on application transition screens in real time. Therefore, we developed an image recognition system that can perform image detection processing at higher speed and with higher accuracy. The system was desired.
[0008] Therefore, the present invention has been devised in consideration of the above-mentioned problems, and is a method for producing a high-speed and high-precision image. To provide an image recognition system and an image recognition program capable of performing image detection processing The purpose is to: [Means for solving the problem]
[0009] In order to solve the above problems, an image recognition system according to the present invention is provided. Data size to reduce the data size of the image and the object image to be detected A reduction unit and a reduction unit for reducing image components included in the search target image whose data size has been reduced. A first matching rate between the reduced component and the object image whose data size has been reduced. a first calculation unit that calculates a match rate; and a reduced structure in which the first match rate exceeds a first threshold. The constituent elements are determined to be provisionally identical images that provisionally match the object image. A first determination unit and a method for extracting an initial component corresponding to the provisional identical image from the search target image. and an extraction unit for calculating a second matching rate which is a matching rate between the initial component and the object image. a second calculation unit for calculating the second matching rate of the initial component that exceeds a second threshold value; and a second determination unit that determines that the image is an identical image that matches the object image. It is characterized by:
[0010] In the image recognition system, the data size reduction unit reduces the search target image and the previous At least one of color reduction processing and downscaling processing may be performed on the object image. preferable.
[0011] In the image recognition system, the first determination unit determines whether the data size is reduced. The first coordinate range, which is the coordinate range of the provisionally identical image in the search target image, is used as a determination result. and outputting the coordinates corresponding to the first coordinate range in the search target image. It is preferable to extract image components in a second coordinate range as the initial components. I wish.
[0012] In the image recognition system, the first determination unit determines whether or not at least one of the first coordinate range is a pixel value. The provisionally identical images that overlap each other are determined to be a single provisionally identical image. It is preferable to output the representative coordinate range as the determination result.
[0013] In the image recognition system, the representative coordinate range is a small portion of the first coordinate range. Among the provisionally identical images that at least partially overlap, the provisionally identical images having the first matching rate equal to or greater than a third threshold value It is preferable that the first coordinate range is of the same image.
[0014] In the image recognition system, the extraction unit extracts at least one of the second coordinate range. The original components that overlap in part are determined to be a single original component, and the representative original component is It is preferable to output the components as the extraction results.
[0015] In the image recognition system, the representative initial component is a component in the second coordinate range. Among the initial components that overlap at least partially, the first matching rate is equal to or greater than a fourth threshold value. It is preferable that the initial component corresponds to the provisional identical image.
[0016] In the image recognition system, the extraction unit extracts a front image of the second coordinate range. It is preferable to set the range to be narrower than the first coordinate range.
[0017] The present invention also provides a method for detecting an object image from a search target image by a computer. The image recognition program includes: data of the search target image and the object image; A data size reduction step of reducing the size of the search target image having the reduced data size. and the reduced components which are image components included in the object with reduced data size. a first calculation step of calculating a first matching rate which is a matching rate with the image of the object; The reduced component having the first matching rate is provisionally matched with the object image. a first determination step of determining that the image is a provisionally identical image; extracting component elements from the search target image; a second calculation step of calculating a second matching rate which is a matching rate with the first image; The initial components with a matching rate of 2 are treated as identical images that match the object image. and a second determination step of determining: Effect of the Invention
[0018] According to the present invention having the above-mentioned configuration, it is possible to perform high-speed and highly accurate image detection processing. It becomes Noh. [Brief description of the drawings]
[0019] [Figure 1] FIG. 2 is a conceptual diagram of an image detection process performed by an image recognition system according to an embodiment of the present invention. [Diagram 2] 1 is a configuration block diagram of an image recognition system according to an embodiment of the present invention. [Diagram 3] FIG. 2 is a functional block diagram of a processor used in the image recognition system according to the embodiment of the present invention. [Figure 4] 4 is a flowchart illustrating an example of processing by the image recognition system according to the embodiment of the present invention. [Diagram 5]1 is a table comparing the image detection speed of an image recognition system according to an embodiment of the present invention with that of a conventional image recognition system. [Figure 6] 1 is a conceptual diagram showing how an object image is detected from a search target image by an image recognition system according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0020] Hereinafter, an image recognition system according to an embodiment of the present invention will be described with reference to the drawings. do.
[0021] FIG. 1 is a diagram showing an example of image detection performed by an image recognition system according to an embodiment of the present invention. As shown in FIG. 1, an image recognition system according to an embodiment of the present invention includes: From the search target image S, which is the search target, the image to be detected (i.e., image recognition is performed) A system for detecting an object image O (a template image for a desired image) It is.
[0022] Specifically, the image recognition system according to the embodiment of the present invention recognizes a target image S of a search. Among various image components, image components that match the object image O are regarded as the same image. This detection process is carried out by detecting whether the search target image S is an image such as a game screen. If the object is constantly changing, the object and each component in the search target image S are The matching rate with the target image O is calculated, and the identity is judged based on the matching rate. The calculation of the match rate and the determination of identity will be described in detail later.
[0023] FIG. 2 is a configuration block diagram of an image recognition system 1 according to an embodiment of the present invention. As shown in the figure, the image recognition system 1 according to the embodiment of the present invention is implemented in, for example, a server. The system includes a processor 11 and a memory 12 including one or more computer-readable media. The image recognition system 1 is equipped with various devices such as a display and a keyboard. It is possible to connect to an external device (not shown) and input from the external device is used for image recognition processing. It is possible to receive images and send the results of image recognition processing to an external device and display them on the device. The image recognition system 1 is connected to an external device by a cable. The communication may be performed by a computer or may be performed over the air using wireless communication.
[0024] The processor 11 performs image recognition processing based on image data input from an external device. It is also possible to transmit the results of image recognition processing to an external device and display them there. The processor 11 can operate based on any operating system (OS). do.
[0025] The memory 12 includes one or more computer-readable media and is executed by the processor 11. Image recognition software 121 and an image data processor for storing image data input from an external device. The data storage unit 122 is provided.
[0026] FIG. 3 shows the function of the processor 11 used in the image recognition system 1 according to the embodiment of the present invention. As shown in FIG. 3, a processor 11 includes image recognition software 12. 1, the data size reduction unit 111, the first calculation unit 112, and the first determination unit 1 13, functions as an extraction unit 114, a second calculation unit 115 and a second determination unit 116.
[0027] The data size reduction unit 111 reduces the data size of the search target image S and the object image O. Specifically, the data size reduction unit 111 reduces the search target image S and the object For the image, the color reduction process reduces the number of colors in the image, and the down-conversion process reduces the number of pixels in the image. At least one of the above is performed.
[0028] The color reduction process averages the R, G, and B information to produce a grayscale image with only brightness. The image is converted to a 56-level grayscale image. This reduces the size of the object image O and enables quick image detection. In the recognition system 1, the user can set whether or not color reduction processing is required, and the amount of color reduction to be performed. The type of processing can be selected arbitrarily.
[0029] The downscaling process reduces the number of pixels in the search target image S and the object image O. A process to reduce the size of the search target image S and the object image O by making them coarse images. In the image recognition system 1, the user can set the downscale You can choose whether or not to process the image and how much to reduce the number of pixels. In downscaling, the fewer the number of pixels, the smaller the data size of the image. This allows for faster image detection, but it is possible to detect fewer pixels than the specified number. If the image is not detected correctly, the image detection accuracy may decrease and proper image detection may not be possible. Depending on the type of the search target image S and the object image O, and the required accuracy and speed of image detection, Therefore, it is necessary to perform downscaling processing within an appropriate range.
[0030] The first calculation unit 112 calculates the search result whose data size has been reduced by the data size reduction unit 111. The reduced components are image components included in the target image S, and the data size reduction unit The first matching rate is the matching rate with the object image O whose data size has been reduced by 111. The calculation of the first matching rate by the first calculation unit 112 is performed using an arbitrary image detection algorithm. This can be done using:
[0031] The first determination unit 113 determines whether the reduced configuration has a first matching rate that exceeds a predetermined threshold (first threshold). The element is determined to be a provisionally identical image that provisionally matches the object image O. Then, the first determination unit 113 determines whether the temporary synchronization is performed for the search target image S whose data size has been reduced. The first coordinate range, which is the coordinate range of one image, is output as the determination result. Based on the horizontal (X-axis direction) and vertical (Y-axis direction) coordinates set for the image, For example, X=300~340, Y=100~150 for the horizontal direction. “Horizontal” refers to a predetermined range based on vertical and horizontal coordinates.
[0032] The extraction unit 114 extracts the initial components corresponding to the provisionally identical image from the search target image S. The extraction of the original components corresponding to the provisional same image is based on the coordinate range in each image. Specifically, the extraction unit 114 performs the first determination in the search target image S. The image constituent elements in the second coordinate range, which is a coordinate range corresponding to the first coordinate range output by Here, the extraction unit 114 extracts the elements as the initial components for the second coordinate range. Set the second coordinate range to be narrower than the first coordinate range. By narrowing the range, i.e. trimming, the file size of the original components is reduced. This allows faster image detection by reducing the size of the initial component. For example, when visual effects such as light and fire, which are often used in games, are generated, This eliminates the visual effect of the pixelated area, making it possible to perform image detection with higher accuracy.
[0033] The second calculation unit 115 calculates the initial components extracted by the extraction unit 114 and the object image. A second coincidence rate, which is the coincidence rate with the image O, is calculated. This second coincidence rate is also calculated by the same method as the first coincidence rate. As with extraction, any image detection algorithm can be used.
[0034] The second determination unit 116 determines whether the initial configuration element has a second matching rate that exceeds a predetermined threshold (second threshold). The second determination unit 1 determines that the image is an image that matches the object image O. 16 is a diagram for the initial components that are determined to be identical images that match the object image O. The position information (X coordinate, Y coordinate) in the search target image S and the search target image S change from moment to moment. In the case of a game screen, etc., in which the initial component is The time information relating to the detected time is extracted. The information and time information is made available to other applications, e.g. automatically. In a game-playing application, an object that acts as an indicator for a specific action is A gate is then generated based on the detected object image O. It will be possible to play the game.
[0035] Next, the operation of the image recognition system 1 having the above-mentioned configuration will be described. A case will be described where an object image O is detected from a search target image S shown in FIG.
[0036] FIG. 4 is a flowchart showing an example of processing by the image recognition system 1 according to the embodiment of the present invention. First, the search target image S and the object image O specified by the user are compared. When the image detection process is started, the data size reduction unit 111 searches for the search target image S and the object The data size of the image O is reduced (step S1). Here, the search target image S and the object image O are subjected to color reduction and downscaling processing. The data size of the search target image S and the object image O is reduced.
[0037] Next, the first calculation unit 112 calculates the data size of the data that has been reduced by the data size reduction unit 111. The reduced components are image components included in the search target image S, and the data size is The first image is a matching rate with the object image O whose data size has been reduced by the reduction unit 111. The matching rate is calculated based on a predetermined image detection algorithm (step S2). Since the data sizes of the image S and the object image O are reduced, the first calculation unit 112 This makes it possible to quickly calculate the first match rate.
[0038] Next, the first determination unit 113 performs a step of determining whether the first matching rate exceeds the first threshold value for the reduced components. Then, it is determined that the image is a provisionally identical image that provisionally matches the object image O (step S 3) The first coordinate range of the provisionally identical image in the search target image S whose data size has been reduced is The determination result is output to the extraction unit 114. The determination may be performed for a plurality of reduced components. If a plurality of reduced components having a first matching rate with the object image O exceeding a first threshold are detected, In this case, the first determination unit 113 outputs the plurality of reduced components as the determination result to the extraction unit 11. It may also be output to 4.
[0039] Next, the extraction unit 114 extracts the first and second images output by the first determination unit 113 from the search target image S. The image components in the second coordinate range corresponding to the first coordinate range of the provisionally identical image are treated as the initial components. Extract it as an element (step S4).
[0040] Next, the second calculation unit 115 calculates the initial components extracted by the extraction unit 114 and the object a second matching rate, which is a matching rate with the target image O and is more accurate than the first matching rate, The calculation is performed based on a specific image detection algorithm (step S5).
[0041] Next, the second judgment unit 116 sets the initial components with the second matching rate exceeding the second threshold as the on It is determined that the image matches the object image O (step S6). When the image is judged, a series of image detection processes by the image recognition system 1 is completed. do.
[0042] FIG. 5 shows images of the image recognition system 1 according to the embodiment of the present invention and a conventional image recognition system. This is a table comparing the detection speed. In the table shown in Fig. 5, the search target image S and Using an object image O, a conventional image recognition system and an image recognition system according to an embodiment of the present invention are compared. The results of image detection performed by the recognition system 1 are shown. The image recognition system 1 according to the embodiment of the present invention uses the same hardware. Conventional image recognition systems use OpenCV's pattern matching algorithm as an image detection algorithm. The ping function is used.
[0043] As shown in FIG. 5, in the image recognition system 1 according to the embodiment of the present invention, It can be seen that the image detection speed is 3 to 4 times faster than the system.
[0044] FIG. 6 shows a process of extracting objects from a search target image by an image recognition system according to an embodiment of the present invention. FIG. 13 is a conceptual diagram showing how a tomographic image is detected.
[0045] In the example shown in FIG. 6, four identical images SI are detected in the search target image S. The image recognition system 1 then detects the same image SI by storing the position information and , and, if necessary, indicates at what point in time the same image SI appears in the image that changes from moment to moment. The position information is generated for each identical image SI, for example, "X The range of X and Y coordinates is 180~210, and the range of Y coordinates is 15~50. The time information is generated as a range, for example, "2 hours 4 hours from the start of the search target image S". The time is generated as the elapsed time from the start to the detection, for example, "12 minutes and 12 seconds after the start." do.
[0046] The position information and time information of the same image SI generated by the image recognition system 1 are For example, it is used for various operations in an application that automatically plays a game. It is used to detect images, allowing the application to automatically play the game. For example, if the image in Figure 6 is a shooting game, the object image O represents the target of attack. In the image shown in FIG. 1, four identical images SI are detected in the search target image S. By monitoring the gameplay, we can check whether the game is working properly. This can be done without requiring game players, making the test quicker and reducing labor costs. This can be done.
[0047] In this way, according to the image recognition system 1 according to the embodiment of the present invention, the search target image S and Perform the first stage of image detection quickly while reducing the data size of the object image O. At the same time, the area to be detected is narrowed down by matching the same image based on the first image detection result. By performing a second highly accurate image detection based on the original image while the original image is in the background, high speed and high accuracy can be achieved. This makes it possible to perform image detection processing at a high speed.
[0048] In the image detection process by the above-mentioned image recognition system 1, the first determination unit 1 As a result of the determination by 13, a plurality of provisionally identical images having at least a part of the first coordinate range overlapping are found. In such a case, all the provisional identical images show the same image components. These provisionally identical images are highly likely to be the same. If the image is processed, it may slow down the image detection process. For the tentative identical images in which at least a portion of the first coordinate range overlaps, a single tentative identical image is The second calculation unit 115 and the second determination unit 116 eliminate duplicated determination results. At this time, the first determination unit 113 may perform processing based on whether or not at least a part of the first coordinate range overlaps. The duplicated provisionally identical images are judged to be a single provisionally identical image, and the representative coordinate range is set By performing processing to output the result of the judgment, it is possible to prevent a decrease in the speed of the image detection processing. Cut.
[0049] Specifically, as the representative coordinate range, for example, a temporary range that at least partially overlaps with the first coordinate range is used. Among the provisionally identical images, provisionally identical images with a first matching rate equal to or higher than a predetermined threshold (third threshold) are The first coordinate range of the image can be used. This allows the second calculation unit 115 and the second decision This enables the processing by the unit 116 to be performed with high accuracy. Among the provisionally identical images having at least a part of the coordinate range overlapping, the provisionally identical image having the highest first matching rate is A first coordinate range of one image may be used, which also allows the second calculator 115 and the second calculator 116 to calculate the coordinates of the first image. This enables the process by the determination unit 116 to be performed with high accuracy.
[0050] In addition, the removal of duplicated judgment results is performed by the extraction unit 114, not the first judgment unit 113. Specifically, the extraction unit 114 extracts the second coordinate ranges that are at least partially overlapping. For the initial component, it is judged to be a single initial component, and the representative initial component is the extraction result. By outputting the image data in this manner, it is possible to prevent a decrease in the speed of the image detection process.
[0051] At this time, as the representative initial component, for example, at least a part of the second coordinate range is overlapped. Among the initial components, provisional matching is performed if the first matching rate is equal to or exceeds a predetermined threshold (fourth threshold). The initial components corresponding to one image can be used. This also makes it possible to use the second calculation unit 11 This allows the processes by the first and second decision units 115 and 116 to be performed with high accuracy. As a constituent element, among the initial components that overlap at least a part of the second coordinate range, the first matching Alternatively, the initial component corresponding to the provisional identical image with the highest rate may be used, which also In particular, the second calculation unit 115 and the second determination unit 116 can perform the processing with high accuracy.
[0052] In addition, for such provisionally identical images in which at least a portion of the first coordinate ranges overlap, Without eliminating the duplicated determination results, all of the duplicated provisionally identical images are calculated by the second calculation unit. The second determination unit 115 and the third determination unit 116 may perform the process. This can improve the accuracy of image detection.
[0053] The present invention is not limited to the above-described embodiment, and various modifications can be made. [Explanation of symbols]
[0054] 1: Image recognition system 11: Processor 12: Memory 121: Image recognition software 122: Image data storage unit 111: Data size reduction section 112: First calculation unit 113:First judgment part 114:Extraction part 115: Second calculation unit 116:Second judgment part O: Object image S: Search target image SI: Same image
Claims
1. The data size of the search target image and the object image to be detected a data size reduction unit for reducing the size of the data; A reduced configuration, which is an image component included in the search target image whose data size has been reduced A first matching rate, which is a matching rate between the element and the object image whose data size has been reduced, is calculated as A first calculation unit that calculates The object is determined for the reduced components that have the first matching rate exceeding a first threshold. a first determination unit that determines that the image is a provisionally identical image that provisionally matches the original image; an extraction unit that extracts an initial component corresponding to the provisional identical image from the search target image; A second matching rate is calculated, which is a matching rate between the initial component and the object image. A calculation unit; The initial components for which the second matching rate exceeds a second threshold are treated as the object image a second determination unit that determines that the images match the same image; An image recognition system comprising:
2. The data size reduction unit performs color reduction processing on the search target image and the object image. The image recognition system according to claim 1 , further comprising at least one of a filter and a downscaling process.
3. The first determination unit determines the provisional identity in the search target image whose data size has been reduced. A first coordinate range, which is a coordinate range of the image, is output as a determination result; The extraction unit extracts a coordinate range corresponding to the first coordinate range in the search target image.
2. The image processing method according to claim 1, wherein image components in a second coordinate range are extracted as the initial components. Image recognition system.
4. The first determination unit determines whether or not the provisionally identical images overlap at least partially in the first coordinate range. The image is determined to be a single provisionally identical image, and the representative coordinate range is output as a determination result.
4. The image recognition system according to claim 3.
5. The representative coordinate range is a range of the provisional identical image that at least partially overlaps with the first coordinate range. The first coordinate range of the provisionally identical image in which the first matching rate is equal to or greater than a third threshold value. The image recognition system according to claim 4.
6. The extraction unit extracts the first and second coordinate ranges from the first and second coordinate ranges. and outputting a representative initial component as an extraction result. Item 4. The image recognition system according to item 3.
7. The representative initial component is the initial component that overlaps at least a portion of the second coordinate range. Among the elements, the initial configuration corresponding to the provisional identical image having the first matching rate equal to or greater than a fourth threshold value 7. The image recognition system according to claim 6, which is an element.
8. The extraction unit determines the second coordinate range to be narrower than the first coordinate range. The image recognition system according to claim 3, wherein the following is set:
9. Image recognition program that allows a computer to detect object images from search target images grams, A data size reduction method for reducing the data sizes of the search target image and the object image. The reduction process and A reduced configuration, which is an image component included in the search target image whose data size has been reduced A first matching rate, which is a matching rate between the element and the object image whose data size has been reduced, is calculated as A first calculation step of calculating The object is determined for the reduced components that have the first matching rate exceeding a first threshold. a first determination step of determining that the image is a provisionally identical image that provisionally matches the original image; an extraction step of extracting an initial component corresponding to the provisional identical image from the search target image; 、 A second matching rate is calculated, which is a matching rate between the initial component and the object image. A calculation process; The initial components for which the second matching rate exceeds a second threshold are treated as the object image a second determination step of determining that the images match the same image; 2. An image recognition program comprising:
Citation Information
Patent Citations
System and method for image recognition using mobile devices
JP2012524343A
