Automatic generation system and method of training images

Abstract

The present invention provides a system and method for automatically generating training images. The present invention adds target images to container images as training images and records the category or location of each target image. During the target addition process, the present invention adds the target image to the target location of the container image as a candidate image, calculates the reliability of the candidate image, and repeats the target addition process until a candidate image with a desired reliability is generated as a training image. The present invention can automatically generate training images, and the recognition difficulty of the training images can be specified by the user, making it suitable for customized recognition training.

Description

Technical Field

[0001] The present invention relates to training images, and in particular to an automatic generation system and method for training images. Background Art

[0002] In order to train artificial intelligence or professional inspectors (such as customs officers conducting contraband inspections) to recognize objects, a large number of samples (training images) must be used, and these samples are not easy to come by.

[0003] To solve the aforementioned problem, a solution for automatically generating samples has been proposed. The aforementioned solution generates a large number of samples through machine learning or random numbers for training.

[0004] However, the results of object recognition training are closely related to the quality of the samples. Once the sample quality is uneven (for example, the differences between samples are too large or too small, or the objects in the samples are too difficult or too easy to identify), the training results will be poor, causing the machine learning model or personnel to only recognize samples of the same type.

[0005] For example, when high-difficulty samples (such as the target is very similar to other objects) are provided for recognition training, the machine learning model or personnel may only recognize high-difficulty samples but fail to recognize low-difficulty samples (such as the target is completely different from other objects).

[0006] Existing solutions for automatically generating samples through machine learning, whether machine learning or random number generation, cannot adjust the recognition difficulty of the generated samples according to user needs and are therefore not suitable for training.

[0007] Therefore, the existing technology for automatically generating samples has the aforementioned problems, and a more effective solution is urgently needed. Summary of the Invention

[0008] The main purpose of the present invention is to provide a system and method for automatically generating training images, which can adjust the recognition difficulty of the generated training images.

[0009] In one embodiment, a method for automatically generating a training image includes the following steps: a) obtaining multiple container images, selecting one from the multiple container images to perform a target addition process to transform the selected container image into a training image including a target image; and b) recording the training image and target data, wherein the target data includes at least one of a target category and a target position of the target image; wherein the target addition process includes the following steps: c1) obtaining the target image; c2) adding the target image to the target position of the selected container image as a candidate image; c3) calculating a reliability of the candidate image, wherein the reliability corresponds to the difficulty of identifying the target image in the candidate image; and c4) when the reliability does not meet a critical value, repeating steps c1)-c3) until the candidate image whose reliability meets the critical value is generated to serve as the training image.

[0010] In one embodiment, a system for automatically generating training images includes a container image source, a target image library, and a control module. The container image source contains multiple container images; the target image library contains multiple target images. The control module is electrically connected to the container image source and the target image library and includes a target addition module, a target recording module, and a reliability calculation module. The target adding module is configured to select one from the multiple container images to perform a target adding process to transform the selected container image into a training image including a target image, wherein the target adding process includes adding the target image to a target position of the container image to serve as a candidate image; the target recording module is configured to record the training image and target data, wherein the target data includes at least one of a target category and the target position of the target image; and the reliability calculation module is configured to calculate a reliability of the candidate image; wherein the control module is configured to repeatedly perform the target adding process via the target adding module when the reliability does not meet a critical value until the candidate image whose reliability meets the critical value is generated to serve as the training image, and the reliability corresponds to the recognition difficulty of the target image in the candidate image.

[0011] In one embodiment, a method for automatically generating a training image includes the following steps: a) obtaining a container image; b) obtaining a target image; c) adding the target image to a target position in the container image to serve as a candidate image; d) calculating a reliability of the candidate image, wherein the reliability corresponds to the difficulty of identifying the target image in the candidate image; and e) when the reliability does not meet a threshold, repeating steps b)-d) until the candidate image with a reliability meeting the threshold is generated to serve as a training image.

[0012] The present invention can automatically generate training images, and the recognition difficulty of the training images can be specified by the user, so it is suitable for customized recognition training. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 FIG. 4 is an architectural diagram of an automatic generation system according to an embodiment of the present invention.

[0014] Figure 2 This is an architectural diagram of an automatic generation system according to another embodiment of the present invention.

[0015] Figure 3 This is an architectural diagram of an automatic generation system according to another embodiment of the present invention.

[0016] Figure 4 This is a partial architecture diagram of an automatic generation system according to another embodiment of the present invention.

[0017] Figure 5 Flowchart of the automatic generation method according to the first embodiment of the present invention.

[0018] Figure 6 This is a partial flow chart of the automatic generation method according to the second embodiment of the present invention.

[0019] Figure 7 This is a partial flow chart of the automatic generation method according to the third embodiment of the present invention.

[0020] Figure 8 This is a partial flow chart of the automatic generation method according to the fourth embodiment of the present invention.

[0021] Figure 9 Flowchart of the automatic generation method according to the fifth embodiment of the present invention.

[0022] Figure 10 A schematic diagram of a target adding process according to an embodiment of the present invention.

[0023] Figure 11 A schematic diagram illustrating a target adding process according to an embodiment of the present invention.

[0024] Figure 12 This is a first schematic diagram of object recognition according to an embodiment of the present invention.

[0025] Figure 13 This is a second schematic diagram of object recognition according to an embodiment of the present invention.

[0026] Figure 14 FIG. 3 is a third schematic diagram of object recognition according to an embodiment of the present invention.

[0027] The description of the accompanying drawings is as follows:

[0028] 1: Automatic generation system

[0029] 10: Control module

[0030] 11: Container image source

[0031] 110: Container Image

[0032] 12: Target Image Library

[0033] 120: Target image

[0034] 13: Output module

[0035] 20: Deformation Database

[0036] 200: Function

[0037] 21: Input module

[0038] 30: Image capture device

[0039] 31: Conveyor belt module

[0040] 32: Display module

[0041] 33: Training Image Library

[0042] 330: Training images

[0043] 331: Target Data

[0044] 40: Storage module

[0045] 400: Computer software

[0046] 401: Object Detection Model

[0047] 402: Generating conditions

[0048] 403: Location selection rules

[0049] 404: Image transformation rules

[0050] 405: Reliability critical value

[0051] 41: Network transmission interface

[0052] 42: Network

[0053] 43: Server

[0054] 50: Target Add Module

[0055] 500: First object detection module

[0056] 501: Target location selection module

[0057] 502: Target selection module

[0058] 503: Image transformation selection module

[0059] 504: Target Positioning Module

[0060] 505: Image generation module

[0061] 506: Second object detection module

[0062] 51: Target Recording Module

[0063] 52: Conveyor belt information acquisition module

[0064] 53: Integrated display control module

[0065] 54: Reliability calculation module

[0066] 60-62: Container Image

[0067] 61': Training footage

[0068] 610-614: Video

[0069] 70-72: Original objects

[0070] 73: Target Image

[0071] 74: Target Data

[0072] 75, 76: Target image after transformation

[0073] 80: Input image

[0074] 800-802: Image block

[0075] 90-91: Range

[0076] 92-94: Object

[0077] CBI: Conveyor Belt Information

[0078] P: Set parameters

[0079] TI: Training Images

[0080] S10-S13: Target adding and recording steps

[0081] S20-S24: Target adding steps

[0082] S30-S34: Target image processing steps

[0083] S40-S42: Reliability calculation steps

[0084] S50-S56: Target adding, recording and displaying steps DETAILED DESCRIPTION

[0085] A preferred embodiment of the present invention is described in detail below with reference to the accompanying drawings.

[0086] During object recognition training, a computer system or user must identify training images including target images (object images, such as contraband or other designated objects) from a large number of images. Currently, there is no way to adjust the recognition difficulty of the target images.

[0087] The present invention provides a method and system for automatically generating training images, which can specify the recognition difficulty (reliability threshold) and automatically generate training images that meet the recognition difficulty, thereby providing object recognition training of different difficulty levels.

[0088] See also Figure 1 and Figure 2 The automatic training image generation system 1 of the present invention may include a container image source 11, a target image library 12, an output module 13, and a control module 10 electrically connected to the aforementioned devices.

[0089] The container image source 11 is used to obtain a plurality of container images 110. Container images 110 (e.g., images of a suitcase, handbag, shopping cart, basket, or other container capable of storing objects) are raw, unprocessed images. The container space may be empty or contain objects (raw objects, such as cosmetics, consumer electronics products, merchandise, goods, or other objects that can be stored in a container).

[0090] The target image library 12 is used to store a plurality of target images 120. The target images 120 are images of targets to be identified in object recognition training (such as pistols, knives, drugs, batteries and other contraband, or other designated objects).

[0091] The output module 13 (e.g., the display module 32, storage module 40, or network transmission interface 41 described below) can be used to output (including storing, displaying, or transmitting to other computers) images. The control module 10 (e.g., a CPU, GPU, processor, controller, SoC, etc.) is used to control the automatic generation system 1 to automatically generate and output training images TI (described in detail below).

[0092] See also Figure 2 In one embodiment, the automatic generation system 1 may include a deformation database 20 and an input module 21 electrically connected to the control module 10 .

[0093] The deformation database 20 stores a plurality of functions 200, each of which is used to perform different image transformation processes (such as rotation, flipping, deformation, brightness adjustment, color adjustment, contrast adjustment, low-pass filtering, high-pass filtering, moving up one layer, moving down one layer, etc.) on the image to transform the image's shape, brightness, color, posture and / or arrangement layer sequence and other attributes.

[0094] The input module 21 (such as a keyboard, mouse, touchpad, touch screen, microphone, etc.) is used for user input operations. For example, the user can input a difficulty setting operation to modify the setting parameter P to adjust the recognition difficulty of the generated training image.

[0095] See also Figure 3 In one embodiment, the container image source 11 may include an image capture device 30 (such as a visible light camera, an X-ray machine, an ultrasonic camera, a thermal camera, etc.), and is used to photograph the physical container to obtain a container image 110 (which may be a color image, a radiographic image such as an X-ray, an ultrasonic image, a thermal image, etc.).

[0096] It is worth noting that when container image 110, target image 120, and training image TI are radiographic images, the pixel value of each pixel in the radiographic image can represent the depth of the object or material density corresponding to each pixel. In other words, control module 10 can color each pixel in the image based on the depth or density of each object in the image, such as using different colors to distinguish locations with different depths or densities, and assigning the same color to locations with the same depth or density.

[0097] Furthermore, the target image 120 and the container image 110 in the target image library 12 and the generated training image TI are of the same type, such as radiographic images or color images.

[0098] In one embodiment, the automatic generation system 1 may include a display module 32 (such as a display such as an LCD or a projector) electrically connected to the control module 10. The display module 32 is used to display images and information.

[0099] In one embodiment, the automatic generation system 1 may include a training image library 33 electrically connected to the control module 10. The training image library 33 may be used to store the training images TI (i.e., training images 330) generated during each training session, and may also store target data 331 for each training image 330. The target data 331 is like an answer to the training image 330, and records relevant information (e.g., object type and / or image embedding position) of the target image 120 included in the training image 330.

[0100] In one embodiment, the automatic generation system 1 may include a conveyor module 31 electrically connected to the control module 10 . The control module 10 may obtain conveyor information CBI from the conveyor module 31 to simulate recognition training for real-time object conveyance inspection (described in detail later).

[0101] See also Figure 4 In one embodiment, the automatic generation system 1 may include a storage module 40 (such as a disk, hard disk, flash memory, RAM, ROM, etc.) electrically connected to the control module 10 and used to store data.

[0102] In one embodiment, the automatic generation system 1 may include a network transmission interface 41 (e.g., a Wi-Fi network module, an Ethernet network module, a cellular network module, etc.) electrically connected to the control module 10. The network transmission interface 41 is used to connect to a network 42 (e.g., the Internet or a local area network) and can connect to a remote server 43 via the network 42.

[0103] In one embodiment, one or more of the container image source 11 , the target image library 12 , the deformation database 20 and the training image library 33 may be network databases, and the control module 10 accesses data in these network databases through the network transmission interface 41 .

[0104] In one embodiment, the control module 10 may include modules 50-54, 500-506 for implementing different functions, the functions of which will be described in detail below. The aforementioned modules 50-54, 500-506 are interconnected (either electrically or electronically) and may be hardware modules (e.g., electronic circuit modules, integrated circuit modules, SoCs, etc.), software modules (e.g., firmware, operating systems, or applications), or a combination of hardware and software modules, without limitation.

[0105] It is worth mentioning that when the aforementioned modules 50-54, 500-506 are software modules, the storage module 40 may include a non-transitory computer-readable recording medium, and the aforementioned non-transitory computer-readable recording medium stores a computer program 400, and the aforementioned computer program 400 records a computer-executable program code. When the control module 10 executes the aforementioned program code, the control functions of the aforementioned modules 50-54, 500-506 can be realized.

[0106] See also Figure 5 The method for automatically generating training images of the present invention may include the following steps.

[0107] Step S10 : The control module 10 obtains a plurality of container images 110 via the container image source 11 . These container images 110 may be ordered (eg, according to the order of luggage conveyance) or unordered, without limitation.

[0108] Executing step S11 : the control module 10 selects at least one of the acquired container images 110 through the object adding module 50 to perform object adding processing, so as to transform the container image 110 into a training image TI including the target image 120 .

[0109] In one embodiment, the storage module 40 stores the generation condition 402 , and the control module 10 selects a portion of the container image 110 to perform the target adding process based on the generation condition 402 .

[0110] For example, the generation condition 402 may be the probability of generating a training image TI (e.g., 0.1%, 10%, 50%, etc.), the total number (e.g., one, two, or one hundred training images TI), the relationship (e.g., the minimum number of container images 110 that must be between at least two training images TI, e.g., 10, 50, or 100 images), etc., without limitation.

[0111] In one embodiment, the user may input a difficulty setting operation to modify the setting parameter P, so as to adjust the recognition difficulty by adjusting the generation condition 402 .

[0112] The present invention uses a mixture of original images (container images without target images) and training images TI (container images with target images added) to perform object recognition training, thereby improving concentration and enhancing training outcomes. Furthermore, the present invention can adjust the concentration level by adjusting the number or frequency of occurrences of the training images TI (generation condition 402).

[0113] In one embodiment, see Figure 6 , the target adding process of the present invention may include the following steps.

[0114] Step S20: The control module 10 selects one or more target images 120 from the target image library 12 via the target selection module 502 to be added to the current container image 110. This selection can be made manually by the user via the input module 21, or automatically by the target selection module 502, such as by random selection or based on a preset selection rule (e.g., selecting an appropriate number (or size) of target images 120 based on the remaining space in the container image 110, or selecting target images 120 of a corresponding difficulty level based on the currently set recognition difficulty level).

[0115] Step S21: The control module 10, through the target adding module 50, adds the selected target images 120 one by one to each target position (predicted generation position) of the current container image 110, and uses the result as a candidate image. The target positions can be manually set by the user or automatically selected by the target adding module 50, without limitation.

[0116] Step S22: The control module 10 calculates the reliability (i.e., reliability score) of the candidate images generated this time through the reliability calculation module 54. The reliability corresponds to the recognition difficulty of the added target image 120 among the candidate images (e.g., the similarity and contrast between the target image 120 and its surrounding images, etc.).

[0117] Step S23 : The control module 10 determines whether the reliability of the candidate image meets a preset reliability threshold 405 (hereinafter referred to as the threshold).

[0118] In one embodiment, if a higher reliability indicates a higher recognition difficulty, the control module 10 determines that the candidate image is in compliance when its reliability is greater than or equal to a threshold value 405. If a lower reliability indicates a higher recognition difficulty, the control module 10 determines that the candidate image is in compliance when its reliability is less than or equal to the threshold value 405. The recognition difficulty of the training image TI generated by the aforementioned method is greater than or equal to the user-set recognition difficulty, and is not overly simple.

[0119] In one embodiment, if the reliability is proportional to the recognition difficulty, the control module 10 may determine that the candidate image is in compliance when the reliability is less than or equal to the threshold value 405. If the reliability is inversely proportional to the recognition difficulty, the control module 10 may determine that the candidate image is in compliance when the reliability is greater than or equal to the threshold value 405. The recognition difficulty of the training image TI generated by the above method is less than or equal to the recognition difficulty set by the user, and is not excessively difficult.

[0120] In one embodiment, the control module 10 may determine compliance only when the reliability of the candidate image is equal to a critical value 405 (which may be a single value or a value range), so that the generated training image TI has a stable recognition difficulty.

[0121] When the reliability of the candidate image does not meet the preset reliability threshold 405 (hereinafter referred to as the threshold), the control module 10 executes steps S20 - S22 again to regenerate the candidate image until the reliability of the generated candidate image meets the threshold 405 .

[0122] In one embodiment, the user may input a difficulty setting operation to modify the setting parameter P, so as to adjust the recognition difficulty by adjusting the threshold value 405 .

[0123] In one embodiment, when adding multiple target images 120 to the same container image 110 , the reliability calculation module 54 calculates the reliability of each target image 120 in the training image TI. The control module 10 only re-executes the adding process for the target images 120 whose reliability does not meet the threshold 405 .

[0124] When the reliability meets the threshold, the control module 10 executes step S24 : the control module 10 sets the candidate image as the training image TI and completes the target adding process.

[0125] Please refer to Figure 5 After completing the target adding process, step S12 is executed: the control module 10 can obtain the target data 331 (which may include the target category and / or the target position) through the target positioning module 504, and record the target data 331 of each target image 120 included in each training image TI in the training image library 33 through the target recording module 51.

[0126] In one embodiment, step S13 may be further performed: the control module 10 records each training image TI in the training image library 33 (ie, the training image 330 ) via the target recording module 51 .

[0127] In one embodiment, the target recording module 51 associates and records each training image and its corresponding target data 331 in the training image library 33 .

[0128] See also Figure 7 In one embodiment, the aforementioned step ( S21 ) of adding the target image 120 to the container image 110 may include the following steps.

[0129] Step S30 : The control module 10 performs object recognition processing on the container image 110 via the first object detection module 500 to detect the object position of each original object in the container image 110 without the target image 120 added.

[0130] In one embodiment, the first object detection module 500 can obtain an object detection model 401. Object detection model 401 can be stored in the storage module 40 or in a cloud server 43 (i.e., cloud computing). Object detection model 401 is generated in advance through machine learning (the generation method is common in the field of machine learning and will not be further described here). The first object detection module 500 performs object recognition on the container image 110 based on object detection model 401.

[0131] Step S31: The control module 10 selects the target images 120 to be added one by one through the target selection module 502. The aforementioned selection can be made manually by the user or automatically by the control module 10.

[0132] Step S32 : The control module 10 sets the target position of each selected target image 120 in the container image 110 based on the preset position selection rule 403 via the target position selection module 501 .

[0133] In one embodiment, the aforementioned position selection rule 403 may be stored in the storage module 40. Furthermore, the user may input a difficulty setting operation to modify the setting parameter P, thereby adjusting the recognition difficulty by adjusting the position selection rule 403 (e.g., overlapping the target image 120 with the original object to increase the difficulty, or placing the target image 120 in a location without the original object to reduce the difficulty).

[0134] In one embodiment, the position selection rule 403 may include that the target position at least partially overlaps with the original object position, the target position is not smaller than the size of the target image 120 , and / or the target position is larger than the size of the original object, etc., without limitation.

[0135] Step S33 : The control module 10 selects at least one function 200 from the deformation database 20 based on the preset image transformation rule 404 via the image transformation selection module 503 to perform image transformation processing on the target image 120 .

[0136] In one embodiment, the image transformation process is to transform the shape, brightness, color, posture, arrangement order (ie, the overlapping relationship between the target image 120 and the original object) and / or other image attributes of the target image 120 .

[0137] In one embodiment, the aforementioned image transformation rules 404 may be stored in the storage module 40. Furthermore, the user may input a difficulty setting operation to modify the setting parameter P, thereby adjusting the recognition difficulty by adjusting the image transformation rules 404 (e.g., increasing the similarity between the target image 120 and the surrounding images to increase the difficulty, or decreasing the similarity between the target image 120 and the surrounding images to decrease the difficulty).

[0138] In one embodiment, the image transformation rule 404 includes selecting a function 200 based on a target category of the target image 120 , selecting a function 200 based on an image difference between the container image 110 and the target image 120 , and / or selecting a function 200 based on a target position and positions of each object, etc., without limitation.

[0139] Step S34 : the control module 10 adds the target image 120 that has undergone at least one image transformation process to the target position of the container image 110 through the image generation module 505 , thereby completing the addition of the target image 120 to the container image 110 .

[0140] The present invention can effectively adjust the recognition difficulty of the generated training image by adjusting the setting parameters P (such as the generation condition 402, the position selection rule 403, the image transformation rule 404 and / or the threshold 405).

[0141] See also Figure 8The control module 10 can use the candidate image as the input image for object recognition processing through the reliability calculation module 54 to obtain the category recognition score and completeness recognition score for each target image in the candidate image, and then calculate the reliability based on these scores. The category recognition score is the probability of the corresponding target category being correctly recognized, and the completeness recognition score is the probability of the corresponding target location being correctly recognized.

[0142] Specifically, the aforementioned step of calculating the reliability ( S22 ) may include steps S40 - S42 .

[0143] Step S40: The control module 10 performs object recognition on each of the multiple image blocks of the input image (e.g., the candidate image) through the second object detection module 506 to determine the probability (first probability) that each image block belongs to each category, assigns the image block to the category with the highest first probability (i.e., assigns the object category to the image block), and sets a category recognition score based on the highest first probability (i.e., assigns the reliability of the image block belonging to the object category).

[0144] See also Figure 12 Candidate image 80 (the luggage image) can be divided into multiple image blocks. Control module 10 can use object detection model 401 to perform object recognition on each image block. For example, the recognition result for image block 800 is "75% probability of being a pistol," the recognition result for image block 801 is "60% probability of being a pistol," and the recognition result for image block 802 is "70% probability of being a mobile phone, 20% probability of being a luggage bag base."

[0145] Next, the control module 10 may determine that image block 800 is a pistol (class recognition score 0.75), image block 801 is a cell phone (class recognition score 0.6), and image block 802 is a cell phone (class recognition score 0.7), thereby completing the calculation of the class recognition score for each image block.

[0146] Please refer to Figure 8 In step S41, the control module 10 uses the second object detection module 506 to perform completeness recognition on multiple ranges of the input image to determine the probability (second probability) that each range covers the complete object, sets the position of the object to the range with the highest second probability (i.e., sets the range of the object), and sets a completeness recognition score based on the highest second probability (i.e., sets the reliability of the range covering the complete object).

[0147] See also Figure 13The control module 10 may select multiple ranges 90-91 in the candidate image 80 and perform completeness recognition on each range using the object detection model 401. For example, the recognition result of range 90 is "there is a 60% probability of covering the complete gun", and the recognition result of range 91 is "there is a 70% probability of covering the complete gun".

[0148] Next, the control module 10 may determine that the completeness identification score of the range 90 is 0.6 and the completeness identification score of the range 91 is 0.7.

[0149] Please refer to Figure 8 Step S42: The control module 10 calculates (such as weighted average, addition or multiplication, etc., not limited to) the reliability of each target image based on the category recognition score and the completeness recognition score of each target image through the reliability calculation module 54.

[0150] In one embodiment, the control module 10 may calculate the reliability according to the following formula (1):

[0151] Reliability = Pr(Class|Grid)*Pr(Object)…………Formula (1) Wherein, Pr(Class|Grid) is the category identification score; Pr(Object) is the completeness identification score.

[0152] See also Figure 14 After executing steps S40-S44, control module 10 obtains the reliability (recognition difficulty) of all objects in the container image. For example, the reliability of target image 92 (pistol) is 0.85, the reliability of original object 93 (cell phone) is 0.98, and the reliability of original object 94 (notebook) is 0.95. Furthermore, control module 10 determines whether the reliability of target image 92 meets a predetermined threshold (e.g., 0.7, 0.9, 0.95, 0.7-0.95, or 0.8-0.9, etc.).

[0153] It is worth mentioning that the first object detection module 500 (step S30 ) may use the object recognition process of steps S40 - S42 to perform object recognition on the original object in the container image 110 .

[0154] In one embodiment, see Figure 3 、 Figure 9 The present invention can provide identification training for real-time object transportation inspection (such as customs luggage inspection).

[0155] The automatic generation system 1 may include a conveyor module 31 electrically connected to the control module 10. The conveyor module 31 is used to transport containers (e.g., luggage). An image capture device 30 (e.g., an X-ray machine) installed at a fixed location can capture each container transported by the conveyor module 31 to sequentially obtain container images 110 (e.g., X-ray images) of different luggage.

[0156] Next, during recognition training, the control module 10 sequentially obtains the captured container images 110 (step S50 ), selects a portion of the container images according to the preset generation condition 402 to add the target image 120 to obtain the training image TI (steps S51 and S52 ), and records the corresponding target data ( S53 ).

[0157] In addition, the control module 10 obtains the conveyor belt information CBI of the conveyor belt module 31 through the conveyor belt information obtaining module 52 (step S54 ), and displays the container image 110 and the training image TI in combination with the conveyor belt information CBI in sequence on the display module 32 through the integrated display control module 53 .

[0158] In one embodiment, the integrated display control module 53 controls the display module 32 to scroll and display (ie, like a ticker display) a plurality of container images and training images according to the order in which the container image 110 and the training image TI are obtained.

[0159] In one embodiment, the scrolling display speed is determined based on the conveying speed of the conveyor belt information. For example, the faster the conveying speed, the faster the scrolling display (the shorter the time each image stays on the display screen, the shorter the time personnel can identify it); the slower the conveying speed, the slower the scrolling display (the longer the time each image stays on the display screen, the longer the time personnel can identify it).

[0160] In one embodiment, the control module 10 does not directly merge the target image 120 and the container image 110 into a single image, but instead implements the training image by simultaneously displaying the target image 120 and the container image 110 on the same screen.

[0161] Specifically, the control module 10 can continuously calculate the latest display position of the target image 120 based on the conveyor belt information, the container image 110 and the target position. When scrolling the corresponding container image 110, the target image 120 is simultaneously displayed at the latest display position at the same scrolling speed, so that the target image 120 moves with the corresponding container image 110, thereby achieving the effect of displaying a training image.

[0162] See also Figure 10-11 The control module 10 sequentially obtains a plurality of container images 60 - 62 and selects the container image 61 to perform target adding processing.

[0163] First, the control module 10 loads a container image (as shown in image 610 ), and performs object recognition on the container image to identify a plurality of original objects 70 - 71 (as shown in image 611 ).

[0164] Next, the control module 10 may select the target position 72 (as shown in the image 612 ), and add the selected target image 73 to the target position 72 (as shown in the image 613 , the target data 74 may also be set at this time).

[0165] Next, the control module 10 may perform image transformation (eg, magnification) on the target image 73 to obtain a target image 75 to improve reliability (as shown in image 614 ).

[0166] If the reliability of the target image 75 of the image 614 (candidate image) does not meet the threshold, the control module 10 may perform image transformation (such as reduction and rotation) on the target image 75 to further improve the reliability (as shown in image 61 ′).

[0167] Finally, if the reliability of the target image 76 of the training image 61 ′ meets the critical value, the control module 10 can replace the container image 61 with the training image 61 ′ to complete the generation of the training image (e.g. Figure 11 ).

[0168] Furthermore, the user can watch the container image 60 , the training image 61 ′, and the container image 60 played in sequence, and perform operations related to the recognition training.

[0169] For example, the user may input a verification operation through the input module 21 to enable the control module 10 to control the display module 32 to display the target data of the target image 76 of the training image 61 ′ (eg, the displayed training image) for user verification.

[0170] In another example, the user may input a stop operation or a run operation through the input module 21 to control the state of the conveyor module 31 to stop or continue running, and change the speed of the conveyor information to stop or run.

[0171] Furthermore, the control module 10 can control the display module 32 to stop scrolling the display screen when the conveyor belt information is set to stop; and can control the display module 32 to continue scrolling the display screen when the conveyor belt information is set to run.

[0172] In this way, the present invention can simulate real-time object conveyance inspection.

[0173] The present invention can automatically generate training images, and the recognition difficulty of the training images can be specified by the user, so it is suitable for customized recognition training.

[0174] The above description is only a preferred embodiment of the present invention, and does not limit the claims of the present invention. Therefore, all equivalent changes made by applying the content of the present invention are similarly included in the scope of the present invention and are hereby stated.

Claims

1. A method for automatically generating training images, comprising the following steps: a) obtaining a plurality of container images, selecting one from the plurality of container images to perform a target addition process to transform the selected container image into a training image including a target image; and b) recording the training image and target data, wherein the target data includes at least one of a target category and a target position of the target image; The target adding process includes the following steps: c1) obtaining the target image; c2) adding the target image to the target position of the selected container image as a candidate image; c3) calculating a reliability of the candidate image, wherein the reliability corresponds to the difficulty of identifying the target image in the candidate image; and c4) When the reliability does not meet a critical value, repeat steps c1) to c3) until the candidate image whose reliability meets the critical value is generated as the training image. Calculating the reliability includes the following steps: e1) using the candidate image as an input image for a second object recognition process to obtain a category recognition score and a completeness recognition score for the target image of the candidate image, wherein the category recognition score is a probability of the target category being correctly recognized, and the completeness recognition score is a probability of the target position being correctly recognized; and e2) calculating the reliability of the target image based on the category recognition score and the completeness recognition score of the target image, The second object recognition process includes the following steps: f1) performing object recognition on each of the plurality of image blocks of the input image to determine a first probability that each of the image blocks belongs to each category, assigning the image block to the category with the highest first probability, and setting the category recognition score based on the highest first probability; and f2) performing completeness recognition on each of the plurality of ranges of the input image to determine a second probability that each range covers a complete object, setting the position of the object to the range with the highest second probability, and setting the completeness recognition score based on the highest second probability; When the candidate image includes a plurality of target images, step c4) further includes repeatedly performing steps c1) to c3) for adding the target image when the reliability of any target image in the candidate image does not meet the critical value.

2. The method for automatically generating training images according to claim 1, wherein: Step c1) further includes selecting the target image from a target image library; and Step c2) further comprises the following steps: d1) performing a first object recognition process on the container image to detect an object position of an original object in the container image; d2) setting the target position in the container image based on a position selection rule; d3) selecting at least one function from a deformation database based on an image transformation rule to perform an image transformation process on the target image to transform at least one of the shape, brightness, color, posture, and arrangement order of the target image; and d4) Adding the processed target image to the target position of the container image.

3. The method for automatically generating training images according to claim 2, wherein: The first object recognition process includes performing object recognition based on an object detection model generated by machine learning; The position selection rule includes at least one of the following: the target position at least partially overlaps with the object position, the target position is not smaller than the size of the target image, and the target position is larger than the size of the original object; The image transformation rule includes at least one of selecting the at least one function based on the target category of the target image, selecting the at least one function based on the image difference between the container image and the target image, and selecting the at least one function based on the target position and the object position.

4. The method for automatically generating training images according to claim 1, wherein: Step a) includes obtaining the plurality of ordered container images and selecting the container image to transform into the training image according to a generation condition. The method further includes the following steps: g1) scrolling and displaying the plurality of container images and the training image in sequence on a display module, wherein the scrolling display speed is determined based on conveyor belt information; and g2) When a verification operation is accepted, the target data of the target image of the training image is displayed.

5. The method for automatically generating training images according to claim 1, wherein: Step a) includes sequentially photographing a plurality of objects via an image capture device to obtain the plurality of container images, and selecting the container images to transform into the training images based on a generation condition. The method further includes the following steps: h1) scrolling and displaying the plurality of container images and the training image in sequence on a display module, wherein a speed of the scrolling display is determined based on conveyor belt information; h2) when the conveyor belt information is set to stop, stopping the scrolling of the display screen of the display module; h3) when the conveyor belt information is set to run, continuing to scroll the display screen; and h4) when receiving a verification operation, outputting the target data of the target image of the displayed training image; The scrolling display of the training image includes continuously calculating the display position of the target image based on the conveyor belt information, the container image and the target position, scrolling and displaying the corresponding container image, and simultaneously displaying the target image at the latest display position so that the target image moves with the corresponding container image.

6. The method for automatically generating training images according to claim 1, further comprising any of the following steps: i) recording each of the training images and the corresponding target data in a training image library; or j) when receiving a difficulty setting operation, adjusting at least one of the threshold value, a generation condition of the training image, a position selection rule of the target position, and an image transformation rule of the target image based on the difficulty setting operation to adjust the recognition difficulty of the target image in the candidate image; wherein, The plurality of container images, the target image, and the training image are radiographic images.

7. A system for automatically generating training images, comprising: a container image source having multiple container images; a target image library having multiple target images; and A control module electrically connected to the container image source and the target image library, the control module comprising: a target adding module configured to select at least one from the plurality of container images and perform a target adding process to transform the selected container image into a training image including a target image, wherein the target adding process includes adding the target image to a target position of the container image as a candidate image; a target recording module configured to record the training image and target data, the target data including at least one of a target category and the target position of the target image; and a reliability calculation module configured to calculate a reliability of the candidate image; The control module is configured to repeatedly execute the target addition process via the target addition module when the reliability does not meet a threshold value until a candidate image with a reliability meeting the threshold value is generated as the training image, wherein the reliability corresponds to the difficulty of identifying the target image with respect to the candidate image. The control module further includes: a second object detection module configured to use the candidate image as an input image for a second object recognition process to obtain a category recognition score and a completeness recognition score for the target image of the candidate image, wherein the category recognition score is a probability of correctly identifying the target category, and the completeness recognition score is a probability of correctly identifying the target location. The reliability calculation module is configured to calculate the reliability of the target image based on the category recognition score and the completeness recognition score. The second object detection module is configured to perform object recognition on multiple image blocks of the input image during the second object recognition process to determine a first probability that each image block belongs to each category, set the image block to the category with the highest first probability, and set the category recognition score based on the highest first probability. The second object detection module is further configured to perform completeness recognition on multiple ranges of the input image to determine a second probability that each range covers a complete object, set the position of the object to the range with the highest second probability, and set the completeness recognition score based on the highest second probability. The control module is configured to execute the target adding process again for the target image when the candidate image includes a plurality of the target images and the reliability of any of the target images of the candidate images does not meet the critical value.

8. The automatic generation system of training images according to claim 7, wherein: The control module also includes: a first object detection module configured to perform a first object recognition process on the container image based on an object detection model to detect an object position of an original object in the container image; a target selection module configured to select the target image from the target image library; a target position selection module configured to set the target position in the container image based on a position selection rule; an image transformation selection module that selects at least one function from a deformation database based on an image transformation rule to perform an image transformation process on the target image to transform at least one of the shape, brightness, color, posture, and arrangement order of the target image; and an image generation module, adding the processed target image to the target position of the container image to generate the training image, The control module is further configured to read the position selection rule, the image transformation rule, the threshold value, and the object detection model generated by machine learning from a storage module; The position selection rule includes at least one of the following: the target position at least partially overlaps with the object position, the target position is not smaller than the size of the target image, and the target position is larger than the size of the original object; The image transformation rule includes at least one of selecting the at least one function based on the target category of the target image, selecting the at least one function based on the image difference between the container image and the target image, and selecting the at least one function based on the target position and each of the object positions.

9. The automatic training image generation system of claim 7, further comprising a training image library electrically connected to the control module; in, The control module is further configured to obtain the plurality of container images in order via the container image source, read a generation condition from a storage module, and select the container image to be transformed into the training image according to the generation condition; The control module further includes: a conveyor belt information acquisition module configured to acquire conveyor belt information; and an integrated display control module configured to scroll and display the plurality of container images and the training image on a display module in sequence, wherein the scrolling display speed is determined based on the conveyor belt information, and configured to display the target data of the target image of the training image when a verification operation is received via an input module; The target recording module is configured to record each of the training images and the corresponding target data in the training image library.

10. The automatic generation system of training images according to claim 7, wherein: The control module is further configured to sequentially photograph a plurality of objects via an image capture device of the container image source to obtain the plurality of container images, read a generation condition from a storage module, and select the container image to be transformed into the training image according to the generation condition; The control module further includes: a conveyor belt information acquisition module configured to acquire conveyor belt information of a conveyor belt module; and an integrated display control module configured to sequentially scroll and display the plurality of container images and the training image on a display module, stop scrolling the display screen of the display module when the conveyor belt information is set to stop, and resume scrolling the display screen when the conveyor belt information is set to run, wherein the scrolling speed is determined based on the conveyor belt information; The control module is further configured to output the target data of the target image of the displayed training image to the display module when receiving a verification operation via an input module; The integrated display control module is further configured to continuously calculate the display position of the target image based on the conveyor belt information, the container image and the target position, scroll and display the corresponding container image, and simultaneously display the target image at the latest display position so that the target image moves along with the corresponding container image.

11. The automatic generation system of training images according to claim 7, wherein: The multiple container images, the target image, and the training image are radiographic images, wherein the control module is further configured to adjust, upon receiving a difficulty setting operation via an input module, at least one of the critical value, a generation condition of the training image, a position selection rule of the target position, and an image transformation rule of the target image based on the difficulty setting operation, so as to adjust the recognition difficulty of the target image in the candidate image.

12. A method for automatically generating training images, comprising the following steps: a) obtaining a container image; b) obtaining a target image; c) adding the target image to a target position of the container image as a candidate image; d) calculating a reliability of the candidate image, wherein the reliability corresponds to the difficulty of identifying the target image in the candidate image; and e) when the reliability does not meet a critical value, repeating steps b)-d) until the candidate image whose reliability meets the critical value is generated as a training image, Calculating the reliability includes the following steps: e1) using the candidate image as an input image for a second object recognition process to obtain a category recognition score and a completeness recognition score for the target image of the candidate image, wherein the category recognition score is a probability of the target category being correctly recognized, and the completeness recognition score is a probability of the target position being correctly recognized; and e2) calculating the reliability of the target image based on the category recognition score and the completeness recognition score of the target image, The second object recognition process includes the following steps: f1) performing object recognition on each of the plurality of image blocks of the input image to determine a first probability that each of the image blocks belongs to each category, assigning the image block to the category with the highest first probability, and setting the category recognition score based on the highest first probability; and f2) performing completeness recognition on each of the plurality of ranges of the input image to determine a second probability that each range covers a complete object, setting the position of the object to the range with the highest second probability, and setting the completeness recognition score based on the highest second probability.

Images