Method and system of training image module

WO2025034260A3PCT designated stage expired Publication Date: 2025-03-13INSTAI INC
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
PCT/US2024/021474
Authority / Receiving Office
WO · WO
Patent Type
Applications
Current Assignee / Owner
Priority Date
2023-08-09
Filing Date
2024-03-26
Publication Date
2025-03-13

AI Technical Summary

Technical Problem

Current AI image recognition models are not suitable for all image recognition hardware modules, especially when the camera resolution is low or used in specific scenarios, leading to misjudgments due to unsuitable pre-trained high-definition models.

Method used

A method and system for training an image module that involves automatically capturing images, classifying them into groups, determining stage transition conditions, and adjusting image collection strategies to generate and update training data for an AI model, ensuring it is optimized for the specific hardware and scenario.

Benefits of technology

This approach enables the derivation of an improved AI model for image recognition that is specifically tailored to the hardware module's capabilities, reducing misjudgments and enhancing the model's performance in specific scenarios.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure US2024021474_13032025_PF_FP_ABST
    Figure US2024021474_13032025_PF_FP_ABST
Patent Text Reader

Abstract

An image module automatically captures images of peripheral objects, collects image frames according to a first image collection strategy, and adds the image frames into an image frame assembly. The image frame assembly is classified into groups of image frames. Indices are generated corresponding to respective numbers of the groups of image frames. Whether a stage transition condition is met is determined according to the indices. Image frames are further collected according to a second image collection strategy when the stage transition condition is met. The image frames are added to obtain an updated image frame assembly. The second image collection strategy includes additionally collecting image frames for prioritized image frames; performing training on an Al model for image recognition according to the updated image frame assembly to derive an improved Al model; and loading the improved Al model into a working image module.
Need to check novelty before this filing date? Find Prior Art

Description

METHOD AND SYSTEM OF TRAINING IMAGE MODULE FIELD OF THE INVENTION

[0001] The present invention relates to method and system of training an image module, and more particularly to method and system of training an image module during a training process of an AI image module that is capable of recognizing images. BACKGROUND OF THE INVENTION

[0002] Up to now, Artificial Intelligence (AI) technology has entered a matured stage, so it can be introduced into industries for applications. However, current AI software and hardware architectures are complex and individual training cost is high. Thus it is adverse to introduce AI technology into small and medium-sized enterprises without always available AI professional engineers. For example, an image recognition engine constructed by artificial intelligence technology can continuously enhance the ability of vehicle recognition through deep learning. However, currently available image data that have been previously collected and preprocessed may not be suitable for some specific and niche purposes. Therefore, a general AI model trained with the above- mentioned ready-made image data is not suitable for all image recognition hardware modules. Especially when the resolution of a camera included in the hardware module is not high enough or it is to be used in a specific scenario, direct application of the ready-made high-definition AI model will lead to misjudgment. As such, it is one of the main objects of the disclosure to ameliorate the drawbacks of the conventional means as mentioned above.SUMMARY OF THE INVENTION

[0003] How to solve the problems encountered in the above-mentioned conventional means is the main object of developing the technical means as disclosed.

[0004] An aspect of the present invention relates to relates to a method of training an image module, adapted to be used in a training process of a specified image module, the method comprising steps of: allocating a first image module for data collection, which automatically captures images of neighboring objects; collecting a first set of image frames for training from the images of neighboring objects captured by the first image module for data collection according to a first image collection strategy, and adding the first set of image frames for training into an image frame assembly for training; classifying the image frame assembly for training into a plurality of groups of image frames, generating a plurality of indices corresponding to respective quantities of the plurality of groups of image frames, and determining whether a stage transition condition is met according to the plurality of indices; collecting a second set of image frames for training from images of neighboring objects captured by the first image module for data collection according to a second image collection strategy when the stage transition condition is met, and adding the second set of image frames for training into the image frame assembly for training to obtain an updated image frame assembly for training, wherein the second image collection strategy comprises additionally collecting image frames for training for at least a prioritized one of the plurality of groups of image frames; performing training on an artificial intelligence model for image recognition according to the updated image frame assembly for training to derive an improved artificial intelligence model for image recognition; and loading theimproved artificial intelligence model for image recognition into the specified image module to form a working image module.

[0005] Another aspect of the present invention relates to a system for training an image module, adapted to be used in a training process of a specified image module. The system includes: an image module for data collection for automatically capturing images of neighboring objects to collect a first set of image frames for training according to a first image collection strategy, and adding the first set of image frames for training into an image frame assembly for training; and an AI training server in communication with the image module for data collection, classifying the image frame assembly for training into a plurality of groups of image frames, generating a plurality of indices corresponding to respective quantities of the plurality of groups of image frames, and determining whether a stage transition condition is met according to the plurality of indices; collecting a second set of image frames for training from images of neighboring objects captured by the image module for data collection according to a second image collection strategy when the stage transition condition is met, and adding the second set of image frames for training into the image frame assembly for training to obtain an updated image frame assembly for training, wherein the second image collection strategy comprises additionally collecting image frames for training for at least a prioritized one of the plurality of groups of image frames; performing training on an artificial intelligence model for image recognition according to the updated image frame assembly for training to derive an improved artificial intelligence model for image recognition; and loading the improved artificial intelligence model for image recognition into the specified image module to form a working image module.

[0006] A further aspect of the present invention relates to a method of training an image module, adapted to be used in a training process of a specified image module. The method comprises: allocating a first image module for data collection, which automatically captures images of neighboring objects according to a first image collection strategy to generate a set of neighboring image frames, based on which amaterial for AI generated theme image frames or a material for AI generated background image frames is regenerated; using the regenerated material for AI generated theme image frames, the regenerated material for AI generated background image frames and an image frame generation model to automatically generate a set of image frames for training, and adding the set of image frames for training into an image frame assembly for training to obtain an updated image frame assembly for training; and performing training on an artificial intelligence model for image recognition according to the updated image frame assembly for training.

[0007] Yet another aspect of the present invention relates to a system for training an image module, adapted to be used in a training process of a specified image module. The system includes: an image module for data collection, automatically capturing images of neighboring objects according to a first image collection strategy to generate a set of neighboring image frames for training, based on which a material for AI generated theme image frames or a material for AI generated background image frames is regenerated; an image frame generation model in communication with the image module for data collection, using the regenerated material for AI generated theme image frames and the regenerated material for AI generated background image frames to automatically generate a set of image frames for training, and adding the set of image frames for training into an image frame assembly for training to obtain an updated image frame assembly for training; and an AI training server in communication with the image frame generation model, performing training on an artificial intelligence model for image recognition according to the updated image frame assembly for training. BRIEF DESCRIPTION OF DRAWINGS

[0008] The invention will become more readily apparent to those ordinarily skilled in the art after reviewing the following detailed description and accompanying drawings, in which:FIG.1 is a flowchart schematically illustrating steps of a method of training an image module according to an embodiment of the present invention; FIG.2 is a functional block diagram schematically illustrating an image module for data collection and an associated training system according to an embodiment of the present invention; FIG.3 is a flowchart schematically illustrating steps of a method of training an image module according to another embodiment of the present invention; FIG.4 is a schematic diagram exemplifying a two-dimensional scatter diagram generated by a method of training an image module according to the present invention; FIG.5 is a flowchart schematically illustrating steps of a method of training an image module according to a further embodiment of the present invention; FIGS.6A, 6B and 6C are schematic diagrams exemplifying a variety of objects generated by a method of training an image module according to the present invention; and FIG.7 is a functional block diagram schematically illustrating an image module for data collection and an associated training system according to another embodiment of the present invention. DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

[0009] The invention will now be described more specifically with reference to the following embodiments. It is to be noted that the following descriptions of preferred embodiments of this invention are presented herein for purpose of illustration anddescription only. It is not intended to be exhaustive or to be limited to the precise form disclosed.

[0010] It should be noted that the terms "first" and "second" in the specification, claims and drawings of the present disclosure are used to distinguish similar items, and are not necessarily used to describe a specific order or sequence. It should be understood that the materials so used are interchangeable under appropriate circumstances, so that the embodiments of the invention described herein can be implemented in other orders than those illustrated or described herein.

[0011] In order to improve the conventional means, a method of training an image module is developed according to an embodiment of the present invention, as illustrated in the flowchart of Figure 1. The method of training an image module according to the present invention can be integrated into a training process of an image module. The method mainly includes steps as follows. One or more image modules for data collection are provided. In this embodiment, a first image module for data collection and a second image modules for data collection are provided (Step 11). The image modules for data collection are used for collecting a plurality of sets of image frames for training, which comply with the same preset specification.

[0012] An image module 20 for data collection, which may be either the first image module for data collection or the second image module for data collection, for example, and an associated training system are schematically illustrated in a functional block diagram of FIG.2. The image module 20 for data collection mainly includes a digital camera unit 201, an image processing unit 202, a memory unit 203 and a wireless signal transmission unit 204. The digital camera unit 201 is used to take a group of image frames for training, which comply with the preset specification. For example, they needto meet a specific image quality, such as pixel size of 1280*720 or 640*480. The image processing unit 202 in communication with the digital camera unit 201 is used to run an AI model 2020 for image recognition. The memory unit 203 in communication with the image processing unit 202 is used to store digital data generated by the digital camera unit 201 and the image processing unit 202. Then the wireless signal transmission unit 204 can send out the group of image frames for training.

[0013] For achieving satisfactory training effects by way of the system and method developed according to the present invention, a user allocates the first image module for data collection at a first specific location for automatically capturing a plurality of images at the first specific location and collecting accordingly a first set of image frames for training, which comply with the preset specification (Step 12). Likewise, the second image module for data collection may be allocated at a second specific location for automatically capturing a plurality of images at the second specific location and collecting accordingly a second set of image frames for training, which comply with the preset specification (Step 13). Training can thus be executed by the AI model for image recognition based on the first set of image frames for training and the second set of image frames for training to derive an improved AI model for image recognition (Step 14). Subsequently, the improved AI model for image recognition is loaded into at least one image module to produce a working image module (step 15). Image frames collected by the working image module would comply with the preset specification. Preferably, the first image module for data collection and the second image module for data collection may have the same hardware specification so that the collected image frames for training can have the same image specification. It would further improve the training effects. Furthermore, when the hardware specification of the first imagemodule for data collection and the second image module for data collection is lower than or equal to the working image module, the improved AI model can be ensured to run correctly in the working image module without deriving problems beyond expectation due to unsatisfactory hardware performance of the working image module. In this embodiment, the AI model for image recognition as described above is an AI model in connection with object detection, which automatically frames a range of a target object, classifies the target object into a specified object, and prompts probability of the specified object that the target object could be.

[0014] For example, the well-trained working image module 22 may be installed on a vehicle for automatic driving assistance, e.g., to be used as an automatic driving assistance device of an electric locomotive or a bicycle. In this case, the two image modules for data collection (of course the number may also be a single one or more than two) may be arranged at different specific positions such as traffic intersections or traffic vehicles for collecting multiple sets of image frames for training. For example, images of a variety of objects appearing at the different specific positions, e.g., various vehicles running across the roads, are picked up to obtain the image frames for training. Afterwards, the collected image frames for training, as schematically shown in FIG. 2, are transmitted from the wireless signal transmission unit 204 included in the image module 20 for data collection to a remote AI training server 29 via a wireless communication network 28 for subsequent training. The remote AI training server 29 is installed therein an AI model 290 for image recognition. The AI model 290 for image recognition performs training to improve subsequent image recognition capability based on the multiple sets of image frames for training, thereby deriving an improved AI model for image recognition. The improved AI model forimage recognition is then loaded into at least one image module, which may have the same or compatible specification as / with the image module 20 for data collection. The at least one image module successfully loaded with the improved AI model for image recognition can thus serve as the working image module 22 for the desired purpose. By having the specification of the image module for data collection and the specification of the working image module the same or compatible, the improved AI model for image recognition can successfully run in the working image module. Therefore, the problem of misjudgment caused by improper specifications of training images in the conventional means can be effectively solved.

[0015] Although the above-described technology facilitates readily collection of images for model training, it is still hard for an ordinary non-professional user to determine whether the quantity and types of the collected images are enough. Therefore, a solution as shown in FIG. 3 is proposed to make further improvement. First of all, an image module 20 for data collection is provided (Step 31). The image module 20 is able to collect image frames for training, which meet a preset specification. The image module 20 for data collection is installed at a proper location to automatically capture images of neighboring objects according to a first image collection strategy to collect a first set of image frames for training, which meet the preset specification (Step 32). Then, a classifying operation may be automatically performed to classify the first set of image frames for training into a plurality of groups of image frames and generate respective indices corresponding to respective quantities of the plurality of groups of image frames. Whether a stage transition condition is met or not is then determined according to the respective indices of the classified groups of image frames (Step 33). In an embodiment, the index of a specified group of imageframes represents an accumulatively counted value of image frames in the specified group of image frames. If it is determined in Step 33 that the stage transition condition is not met, go back to Step 32 to continue collection of image frames for training. On the other hand, if it is determined in Step 33 that the stage transition condition is met, proceed to Step 34 to switch from the first image collection strategy is switched to the second image collection strategy. According to the second image collection strategy, images of neighboring objects are further captured to update the first set of image frames for training. The second image collection strategy includes additionally collecting image frames for training for at least a prioritized one of the plurality of groups of image frames. For example, supplementarily capturing image frames for one of the classified groups of image frames having the least number of image frames for training is automatically performed (Step 34). Accordingly, an AI model for image recognition can be improved with at least the first set of image frames to generate an improved AI model for image recognition (Step 35). Subsequently, the improved AI model for image recognition is loaded into at least one image module to serve as the working image module for collecting image frames, which meet the preset specification (Step 36). In an embodiment, each of the images collected by the image modules for data collection is tagged with a device tag and / or a spot tag. The device tag indicates which image module captures the specified image. The spot tag indicates where the specified image is captured. After the prioritized one of the plurality of groups of image frames is determined, e.g., the one having the least number of image frames for training, the second image collection strategy is to additionally collect image frames for training for that group of image frames by a designated image module and / or at a designated location according to the tags. For example, but not limited to thefollowing, if a majority of the image frames in the prioritized group are collected by specific image module or modules, the second image collection strategy may be executed by the specific image module or modules only, Alternatively, if a majority of the image frames in the prioritized group are collected at a specific location, the second image collection strategy may be executed only by the image module or modules installed at the specific location.

[0016] The main principle of the solution is to implement different image collection strategies at different stages of the automatic image-collection process performed with cooperation between the image module 20 for data collection and AI training server 29. When a stage transition condition is met, a first image collection strategy is switched to a second image collection strategy. For example, the first image collection strategy is to collect images of all the objects appearing in a practical scene and accumulate the number of images as quickly as possible. The AI training server 29 automatically determines whether the number of collected images reaches a level meeting the stage transition condition. If the stage transition condition is met, a basic model training can be started. The second image collection strategy is to require supplementary image collection from the image module 20 for data collection for training.

[0017] For example, at first, the remote AI training server 29 performs numerical analysis of at least two parameters on a large number of collected images. For example, an absolute height of a vehicle in a collected image can be regarded as a first parameter, and an absolute length of the vehicle in the collected image can be regarded as a second parameter for classification. The first parameter and the second parameter are combined into a two-dimensional coordinate to indicate the collected image. Then aplurality of collected images are classified. Furthermore, each of the classified images is indicated in a two-dimensional scatter diagram as shown in FIG.4, which show min- max normalized values of height and length. It would be understood that if a third parameter or more parameters are defined, each collected image can be located in a three-dimensional or more dimensional scatter diagram.

[0018] As can be seen from the two-dimensional scatter diagram shown in FIG.4, by way of this kind of numerical analysis method, vehicles are automatically classified in a manner that the vehicles having similar sizes, e.g., similar heights and / or lengths, are automatically located in the same zone of the two-dimensional scatter diagram. After classification, the AI training server 29 may count the samples shown in the two- dimensional scatter diagram to realize numbers of images in connection with respective sizes of vehicles. Based on the numbers of grouped images in connection with respective sizes of vehicles, whether a stage transition condition concerning the first image collection strategy is met or not can be determined. Furthermore, the classified images may further be analyzed based on image galleries. For example, the advanced analysis can be automatically executed by the artificial intelligence training server 29 or manually selected and marked by well-trained human being. Similar and redundant image or images for each classified group may be eliminated, or presentative image or images for each classified group may be directly screened out and provided to the model for training.

[0019] Of course, the abscissa axis and the ordinate axis of the two-dimensional scatter diagram may be defined as other physical parameters instead of the above- mentioned absolute height and length of vehicles. For example, the parameters may be average brightness of images, complexity of scenes of images, shooting angles ofimages and so on. In this way, the collected images can be classified according to different needs. For example, day and night can be distinguished according to brightness; city / country or indoor / outdoor can be distinguished according to complexity of scenes; and overlooking and eye-level looking can be distinguished according to shooting angles. The classified and counted results are finally provided for the system to automatically determine whether the stage transition condition is met or not as illustrated in FIG.3. In an alternative embodiment, the quantities of the image frames in the classified groups are not directly counted and used as the indices. Instead, after the classifying operation, the quantities of the image frames in the classified groups are shown in a manner of densities in the scatter diagram. Indices corresponding to the respective quantities of the classified groups of image frames are referred to for determining whether the stage transition condition is met or not. For example, the stage transition condition is met when the density of each of the classified groups of image frames in the scatter diagram reaches a minimum threshold.

[0020] Furthermore, in order to collect image frames for training more efficiently, after the AI training server 29 receives the collected image frames for training, the collected image frames are not only used for training, but also used for determining whether to proceed to the second image collection strategy, i.e., determining whether the stage transition condition is met or not. If the second image collection strategy is entered, a request for supplementary image collection is sent to the image module for data collection. In this stage, the image module for data collection collects only specific image frames complying with specific requirements in order to quickly collect a sufficient amount of the specific image frames for training. For example, the second image collection strategy may include automatically picking up image framesbelonging to specific classified group(s), which has the fewest number(s) of image frames for training. In addition, concerning classified group(s) with poor training performance on specified model(s), the second image collection strategy may include automatically picking up image frames belonging to the specific classified group(s), whose performance in previous model training is dissatisfactory, thereby speeding up the overall model training.

[0021] Taking vehicles as an example, the AI training server 29 first determines the specific classified groups of image frames with the least numbers, e.g., trucks, ambulances, fire engines and / or sprinklers. Then the AI training server 29 informs the image module for data collection of respective feature data, such as sizes and colors, of the specific classified groups of vehicles. Alternatively, the AI training server 29 may have a prompt shown on a display of the image module for data collection to remind a user of missing data of certain vehicles, so the user can relocate the image module to a place where a relatively large number of such vehicles may pass. In this way, the image module for data collection can rapidly collect image frames that meet the specific requirements such as specific sizes, colors, etc. The supplementarily collected image frames are then sent by the image module back to the AI training server 29 to be added into the specific classified group(s). In this way, the image collection can be more efficiently, and meanwhile, subsequent machine learning and training improvement can also be more efficiently.

[0022] In order to ensure whether the AI model 290 for image recognition has completed training improvement, the method of training an image module may further include the following steps. Once the image recognition capability of the AI model 290 for image recognition has been improved to a predetermined level, the AI trainingserver 29 loads the AI model 290 for image recognition into the image module to function as the working image module 22. For example, in a case that the AI model 290 for image recognition is used for classification of vehicles, the image recognition capability can be determined to achieve the predetermined level if the accuracy of image recognition for classification exceeds a predetermined rate, e.g., 95%. In another case that the AI model 290 for image recognition is used for reading variable numbers of a charging device, such as electric meter, water meter or gas meter, the image recognition capability can be determined to achieve the predetermined level if the accuracy of image recognition for characters exceeds a predetermined rate, e.g., 99.9%.

[0023] Please refer to FIG.5, which is a flowchart schematically illustrating steps of a method of training an image module according to a further embodiment of the present invention. The method is used during the training process of the image module, and includes the following steps. First of all, a first image module for data collection is installed (Step 51). The first image module for data collection automatically captures neighboring images according to a first image collection strategy to accumulatively collect a group of neighboring image frames, each of which may be a theme image frame to be added to an existing group of theme image frames or a background image frame to be added to an existing group of background image frames. (Step 52). The group of theme image frames and the group of background image frames are used as material for generating the image frames for training. In detail, the first image collection strategy in this embodiment may include collecting images of objects appearing in practical scenes to rapidly accumulate the number of images, and may further include collecting images in empty scenes without objects in a variety of time slots and weather conditions.

[0024] As mentioned above, the group of neighboring image frames are added to two groups of material for generating the image frames for training. One of them is a group of theme image frames. Taking a scene captured from a traffic vehicle for example, the group of theme image frames would focus on a variety of theme objects, e.g., a vehicle 61 as illustrated in FIG. 6A, a vehicle 62 as illustrated in FIG. 6B or a pedestrian as illustrated in FIG.6C. The other one is a group of background image frames. Taking a scene captured from a traffic vehicle for example, the group of background image frames may include traffic scenes such as various intersections or railroad crossings. In brief, the neighboring image frames collected in this embodiment are classified into at least two kinds of image frames, one directed to theme image frames and one directed to background image frames. The generation of more image frames for training is then performed by combining the image frames.

[0025] Accordingly, the group of theme image frames and the group of background image frames are used as the material for generating a group of image frames for training by an image frame generation model 70 as illustrated in FIG.7 (Step 53). The group of image frames for training are integrated into an image frame assembly for training to obtain an updated image frame assembly for training (Step 54). Subsequently, an AI model for image recognition can be trained with the AI model for image recognition (Step 55).

[0026] Taking again a scene captured from a traffic vehicle, for example, the above- mentioned image frame generation model 70, e.g., currently popular image processing programs or generative AI model, may combine background image frames, e.g., a variety of traffic scenes such as crossroads or railroad crossings without vehicles or pedestrians, with theme image frames, e.g., a variety of vehicles or pedestrians, togenerate a large number of different image frames for training, which meet a preset specification. For example, the resolution of image frames meets requirement. Examples of combination include the same vehicle, e.g., a truck or a bus, in different backgrounds, e.g., crossroads in a city or a railroad crossing in a suburb, or different vehicles in the same background. In this way, a large number of image frames, which meet a preset specification, can be generated in a short term. Moreover, since the images frames are automatically generated by way of image frame generation model, labels or tags may be automatically added to the image frames for training to be automatically interpreted and classified.

[0027] Furthermore, in order to improve recognition capability, a method of generating image frames for training with different contents and styles may be introduced into the method according to the present invention. For example, the image frame generation model 70 in communication with the image module 20 for data collection and the AI training server 29 may derive a plurality of image frames for training with the same background image frame. For example, the same vehicle may be combined with the same background image frames in different time slots, e.g., night, dawn, noon or dusk, and different weather conditions, e.g., being sunny, cloudy, rainy or snowy. In this way, even if the image frames collected in the practical scene are not enough, the collected image frames may be used as material for AI generated image frames to generate a large number of diverse image frames with different image styles for training. Moreover, an image frame of a theme object, e.g. a vehicle, may derive a plurality of theme image frames as viewed in different directions or at different angles. Accordingly, a large number of image frames for training can be obtained from a limited number of image frames actually collected. According to the present invention,a variety of AI models for image recognition may be used in the present invention, For example, foundational models (FM) adopting different machine learning algorithms may be used for training, and it is understood that different machine learning algorithms would result in AI models with different features, such as generalization- oriented or specialization-oriented AI models. Meanwhile, different types of data may be required for training with different types of AI models. Furthermore, the same type of AI model may operate in image modules for data collection (or working image modules) with different hardware specifications. Since data required for training with different types of AI foundational models and / or different hardware specifications of image modules may be different, the weights of various images included in the training data may be adjusted according to the type of the AI foundational model and / or the hardware specification of the image module. For example, for a specialization-oriented AI foundational model, which has relatively low generalization capability, the weights of specific images directed to specific features out of the training data are raised. Accordingly, more images having the specific features would be collected or generated during the training process. With a large quantity of images having the specific features for training, satisfactory recognition capability of the specialization-oriented AI model after training can be obtained.

[0028] While the invention has been described in terms of what is presently considered to be the most practical and preferred embodiments, it is to be understood that the invention needs not be limited to the disclosed embodiment. On the contrary, it is intended to cover various modifications and similar arrangements included within the spirit and scope of the appended claims which are to be accorded with the broadest interpretation so as to encompass all such modifications and similar structures.

Claims

WHAT IS CLAIMED IS:

1. A method of training an image module, adapted to be used in a training process of an image module, the method comprising steps of: allocating a first image module for data collection, which automatically captures images of peripheral objects; collecting a first set of image frames for training from the images of peripheral objects captured by the first image module for data collection according to a first image collection strategy, and adding the first set of image frames for training into an image frame assembly for training; classifying the image frame assembly for training into a plurality of groups of image frames, generating a plurality of indices corresponding to respective quantities of the plurality of groups of image frames, and determining whether a stage transition condition is met according to the plurality of indices; collecting a second set of image frames for training from images of peripheral objects captured by the first image module for data collection according to a second image collection strategy when the stage transition condition is met, and adding the second set of image frames for training into the image frame assembly for training to obtain an updated image frame assembly for training, wherein the second image collection strategy comprises additionally collecting image frames for training for at least a prioritized one of the plurality of groups of image frames; performing training on an artificial intelligence model for image recognition according to the updated image frame assembly for training to derive an improved artificial intelligence model for image recognition; andloading the improved artificial intelligence model for image recognition into a working image module.

2. The method of training an image module according to claim 1, wherein the first image module for data collection or an AI training server is used to collect the first set of image frames for training according to the first mage collection strategy, classify the image frame assembly for training and collect the second set of image frames for training according to the second mage collection strategy, and an image frame collected by the working image module, the first set of image frames for training and the second set of image frames for training all meet a preset specification.

3. The method of training an image module according to claim 2, further comprising: automatically loading the improved artificial intelligence model for image recognition into the specified image module to form the working image module when an image recognition capability of the improved artificial intelligence model for image recognition exceeds a predetermined level.

4. The method of training an image module according to claim 3, wherein the image recognition capability of the improved artificial intelligence model for image recognition is determined to exceed the predetermined level when an accuracy of image classification exceeds a predetermined value.

5. The method of training an image module according to claim 3, wherein the image recognition capability of the improved artificial intelligence model for image recognition is determined to exceed the predetermined level when an accuracy of image recognition for characters exceeds a predetermined value.

6. The method of training an image module according to claim 1, further comprising: providing a second image module for data collection, the second image module for data collection having the same level of hardware specification as the first image module for data collection and a lower or the same level of hardware specification than the working image module, wherein: the second image module for data collection is allocated to automatically capture images of neighboring objects and collect a third set of image frames for training according to the first image collection strategy, and the third set of image frames for training is added into the image frame assembly for training to obtain the updated image frame assembly for training; and when the stage transition condition is met, the second image module for data collection automatically captures images of neighboring objects and collect a fourth set of image frames for training according to the second image collection strategy, and the fourth set of image frames for training is added into the image frame assembly for training to obtain the updated image frame assembly for training.

7. The method of training an image module according to claim 6, further comprising tagging the images collected by the first image module for data collection and the second image module for data collection, wherein each tag of each specified image includes a device tag indicating the first image module for data collection or the second image module for data collection, by which the specified image is captured, and / or a spot tag indicating a location where the specified image is captured, and the second image collection strategy is to additionally collect image frames for training for the prioritized one of the plurality of groups of image frames by adesignated image module for data collection and / or at a designated location according to the each tag.

8. The method of training an image module according to claim 6, wherein each of the first image module for data collection and the second image module for data collection comprises: a digital camera unit, an image processing unit, a memory unit and a wireless signal transmission unit, wherein the digital camera unit is used to take a group of image frames for training, which comply with the preset specification, the image processing unit in communication with the digital camera unit is used to run an AI model for image recognition, the memory unit in communication with the image processing unit is used to store digital data generated by the digital camera unit and the image processing unit, and the wireless signal transmission unit is use for transmitting out the group of image frames for training.

9. The method of training an image module according to claim 1, wherein the first data collection strategy is to collect images of all objects appearing in a practical scene and accumulatively count the collected images in the plurality of groups of image frames to obtain a plurality of accumulatively counted values as the plurality of indices, respectively, and the stage transition condition is met when an accumulatively counted value of the collected images in each of the plurality of groups of image frames reaches a minimum threshold, and a difference between an average value and the accumulatively counted value of the collected images in each of the plurality of groups of image frames is less than a predetermined value.

10. The method of training an image module according to claim 8, wherein the second image collection strategy is to automatically capture supplementary image framesfor training for one of the plurality of groups of image frames, which has the least accumulatively counted value of collected images.

11. The method of training an image module according to claim 1, wherein the second image collection strategy is to automatically capture supplementary image frames for training for one of the plurality of groups of image frames, which has dissatisfactory performance in previous model training.

12. The method of training an image module according to claim 1, wherein the first data collection strategy is to collect images of all objects appearing in a practical scene and estimate densities of the plurality of groups of image frames in a scatter diagram as the plurality of indices, respectively, wherein each of the plurality of groups of image frames includes a plurality of parameters respectively corresponding a plurality of coordinates of the scatter diagram, and the stage transition condition is met when a density of each of the plurality of groups of image frames in the scatter diagram reaches a minimum threshold.

13. A system for training an image module, adapted to be used in a training process of a specified image module, the system comprising: an image module for data collection for automatically capturing images of neighboring objects to collect a first set of image frames for training according to a first image collection strategy, and adding the first set of image frames for training into an image frame assembly for training; and an AI training server in communication with the image module for data collection, classifying the image frame assembly for training into a plurality of groups of image frames, generating a plurality of indices corresponding torespective quantities of the plurality of groups of image frames, and determining whether a stage transition condition is met according to the plurality of indices; collecting a second set of image frames for training from images of neighboring objects captured by the image module for data collection according to a second image collection strategy when the stage transition condition is met, and adding the second set of image frames for training into the image frame assembly for training to obtain an updated image frame assembly for training, wherein the second image collection strategy comprises additionally collecting image frames for training for at least a prioritized one of the plurality of groups of image frames; performing training on an artificial intelligence model for image recognition according to the updated image frame assembly for training to derive an improved artificial intelligence model for image recognition; and loading the improved artificial intelligence model for image recognition into the specified image module to form a working image module.

14. A method of training an image module, adapted to be used in a training process of a specified image module, the method comprising steps of: allocating a first image module for data collection, which automatically captures images of neighboring objects according to a first image collection strategy to collect a set of neighboring image frames, each of which is a theme image frame to be added to a group of theme image frames or a background image frame to be added to a group of background image frames; using the group of theme image frames and / or the group of background image frames as material for generating a set of image frames for training by an imageframe generation model, and adding the set of image frames for training into an image frame assembly for training to obtain an updated image frame assembly for training; and performing training on an artificial intelligence model for image recognition according to the updated image frame assembly for training.

15. The method of training an image module according to claim 14, wherein the first image collection strategy comprises: collecting images of objects appearing in a specified scene; and collecting images in an empty scene without objects in a variety of time slots and weather conditions to generate the set of neighboring image frames.

16. The method of training an image module according to claim 14, wherein the image frame generation model uses the group of theme image frames and the group of background image frames to automatically generate the set of image frames for training, which comply with the preset specification, and the image frame generation model further automatically adds labels or tags to the set of image frames for training to perform automatic interpretation and classification.

17. The method of training an image module according to claim 14, wherein the image frame generation model uses the group of theme image frames, the group of background image frames and a plurality of image style files to automatically generate the set of image frames for training, thereby generating a large number of diverse image frames for training, and the image frame generation model further automatically adds labels or tags to the set of image frames for training to perform automatic interpretation and classification.

18. A system for training an image module, adapted to be used in a training process of a specified image module, the system comprising: an image module for data collection, automatically capturing images of neighboring objects according to a first image collection strategy to collect a set of neighboring image frames for training, each of which is a theme image frame to be added to a group of theme image frames or a background image frame to be added to a group of background image frames; an image frame generation model in communication with the image module for data collection, using the group of theme image frames and the group of background image frames to automatically generate a set of image frames for training, and adding the set of image frames for training into an image frame assembly for training to obtain an updated image frame assembly for training; and an AI training server in communication with the image frame generation model, performing training on an artificial intelligence model for image recognition according to the updated image frame assembly for training.

19. The system of training an image module according to claim 18, wherein the artificial intelligence model for image recognition is trained based on a generalization-oriented foundational model.

20. The system of training an image module according to claim 18, wherein the artificial intelligence model for image recognition is trained based on a specialization-oriented foundational model, in which a specific image directed to a specific feature has a higher weight out of images in the updated image frame assembly for training.

Citation Information

Patent Citations

  • Drone based capture of multi-view interactive digital media

    US20200213578A1

  • Building units for machine learning models for denoising images and systems and methods for using same

    US20220309618A1

  • Image frame super-resolution implementation method and apparatus

    US20230162323A1