Image classification model training method and image recognition method

By automatically extracting image features and model information of automobile parts in the production system and training the image classification model, the problems of long waiting time and low efficiency in the existing technology are solved, and fast and accurate component identification and classification are achieved. Improve work efficiency and data quality.

CN115115866BActive Publication Date: 2024-11-22BMW BRILLIANCE AUTOMOTIVE
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
CN202110292427.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-03-18
Publication Date
2024-11-22
Estimated Expiration
2041-03-18

AI Technical Summary

Technical Problem

The existing methods of auto parts identification and classification have problems such as long waiting time for identification, poor recognition effect, and slow inference speed. It also requires a lot of manual intervention, which increases repetitive work and time-consuming, making it difficult to achieve intelligent tracking and data review.

Method used

By acquiring product images, automatically determine the model and part models using the information in the production system, extract sample images from the image as training data, input them into the image classification model, train a model for identifying part models, and live on the production line This model is used for identification.

Benefits of technology

It greatly saves recognition time, improves work efficiency, shortens training and inference speed, improves the sensitivity and accuracy of classification models, supports efficient classification and traceability of component data, reduces costs, and is suitable for the identification of various product parts. and classification.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115115866B_ABST
    Figure CN115115866B_ABST
Patent Text Reader

Abstract

The present disclosure provides a training method for an image classification model and an image recognition method. The method includes: acquiring an image of a product; receiving information from a production system of the product to determine a first parameter representing a model of the product and a second parameter representing a model of a part of the product; determining a sample image corresponding to the part from the image of the product according to the first parameter and the second parameter; and inputting the sample image and the second parameter as training data into an image classification model to obtain an image classification model for recognizing the part model of the product.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to a method for training an image classification model and an image recognition method using the image classification model. Background Art

[0002] In the era of intelligent production management of Industry 4.0, systems based on the integration of big data and industrial Internet (sensors) are used on a large scale in production, and manufacturing companies are beginning to develop in the direction of intelligent, agile, and visualized flexible production and manufacturing. As the foundation of the automotive industry, automotive parts are a necessary factor to support the sustainable and healthy development of the automotive industry.

[0003] Due to the dispersion of network information, data integration based on different data sources is currently a very important application field. Object recognition is closely related to applications in the information field. Different application systems will have different object recognition optimization tasks, and there are many categories of automotive parts, such as engine parts, transmission system parts, brake system parts, steering system parts, walking system parts, electrical instrument system parts, automotive lamps, automotive modification, security and anti-theft, automotive interior, automotive exterior, comprehensive accessories, audio-visual appliances, chemical care, car body and accessories, etc. The effectiveness of object recognition determines the quality of the global database formed after the integration of parts data. Therefore, the identification and classification of automotive parts has become an important technical requirement. Summary of the invention

[0004] The inventors have discovered that existing methods for identifying and classifying automobile parts have many problems.

[0005] For example, auto parts categories include engine accessories, transmission accessories, brake accessories, steering accessories, running accessories, electrical instrument accessories, auto lamps, auto modification, security and anti-theft, auto interior, auto exterior, comprehensive accessories, audio and video appliances, chemical care, car body and accessories, etc. There are many parts parameters, which can easily lead to problems such as long recognition waiting time, poor recognition effect, slow inference speed, slow training speed, etc.

[0006] In addition, when performing identification and classification, it is necessary to manually determine the extracted features and label the types, which increases a lot of repetitive manual work, is time-consuming and labor-intensive, and is not conducive to improving work efficiency. Take the automobile wheel hub as an example. The wheel hub is an important component of automobile parts. During the production, manufacturing and transportation process, the wheel hub needs to be classified and identified according to the actual situation. The existing wheel hub recognition system uses traditional computer vision algorithms, which is time-consuming and labor-intensive.

[0007] Moreover, existing methods cannot effectively implement intelligent tracking and data review of recognition results, and are insufficient to support complex parts management needs.

[0008] Embodiments of the present invention can provide solutions to one or more of the above problems. Moreover, these solutions are not only applicable to automotive parts, but can also be extended to the identification and classification of various product parts.

[0009] According to a first aspect of an embodiment of the present invention, a method for training an image classification model is provided, comprising: acquiring an image of a product; receiving information from a production system of the product to determine a first parameter representing a model of the product and a second parameter representing a model of a part of the product; determining a sample image corresponding to the part from the image of the product based on the first parameter and the second parameter; and inputting the sample image and the second parameter as training data into an image classification model to obtain an image classification model for identifying the part model of the product.

[0010] According to a second aspect of an embodiment of this document, there is provided an image recognition method using an image classification model, comprising: acquiring an image of a product; receiving information from a production system of the product to determine a first parameter representing a model of the product; determining an image of a part to be detected from the image of the product according to the first parameter; and inputting the image of the part to be detected into an image classification model to determine the model of the part to be detected according to the image classification model.

[0011] According to a third aspect of the embodiments of this document, there is provided a computing device, comprising: a memory on which instructions are stored; and a processor configured to execute the instructions to perform the training method according to the aforementioned first aspect.

[0012] According to a fourth aspect of the embodiments of this document, there is provided a computing device, comprising: a memory on which instructions are stored; and a processor configured to execute the instructions to perform the image recognition method according to the aforementioned second aspect.

[0013] According to a fifth aspect of the embodiments of this document, there is provided an image classification training system, comprising: an imaging device configured to capture an image of a product in response to a signal of a production system; and a computing device according to the aforementioned third aspect.

[0014] According to a sixth aspect of the embodiments of this document, there is provided an image recognition system comprising: an imaging device configured to capture an image of a product in response to a signal of a production system; and a computing device according to the aforementioned fourth aspect.

[0015] According to the seventh aspect of the embodiments of this document, a computer-readable storage medium is provided, on which instructions are stored. When the instructions are executed by a computer, the computer executes the training method according to the aforementioned first aspect or the image recognition method according to the entire second aspect.

[0016] One of the advantages of the embodiments of the present disclosure is that, through integration with the production system, part images can be automatically marked based on information obtained from the production system, which can greatly save time and greatly help improve work efficiency.

[0017] Another advantage of the embodiments of the present disclosure is that, compared with the recognition parameters in the prior art, the classification parameters used in the present method are smaller in size, have a shorter classification waiting time, and have faster training and inference speeds.

[0018] Another advantage of the embodiments disclosed herein is that the classification model has a high sensitivity and provides a variety of flexible deployment methods, which reduces the difficulty of implementation. It can provide an interface for the business to facilitate the business to classify, query, trace and discover problems in component data, thereby helping to optimize business processes.

[0019] Another advantage of the embodiments disclosed herein is that the cost is controllable, and the high-precision model construction and automatic classification are highly generalizable, which has long-term significance for improving the quality of the global database formed after the parts data is integrated.

[0020] Other features and advantages of the present invention will become more apparent from the following detailed description of exemplary embodiments of the present invention with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings, which constitute a part of the specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0022] The present disclosure may be more clearly understood from the following detailed description with reference to the accompanying drawings, in which:

[0023] Figure 1 A schematic diagram showing a processing flow of training an image recognition model according to the prior art.

[0024] Figure 2 A schematic diagram of a processing flow for training an image classification model according to an embodiment of the present disclosure is shown.

[0025] Figure 3 A schematic diagram of an image classification training system according to an embodiment of the present disclosure is shown.

[0026] Figure 4 A flowchart of a method for training an image classification model according to an embodiment of the present disclosure is shown.

[0027] Figure 5 A schematic diagram of an image recognition system according to an embodiment of the present disclosure is shown.

[0028] Figure 6A flowchart of an image recognition method using an image classification model according to an embodiment of the present disclosure is shown.

[0029] Figure 7 An exemplary configuration of a computing device that can implement an embodiment according to the present disclosure is shown.

[0030] For ease of understanding, the position, size, range, etc. of each structure shown in the drawings and the like may not represent the actual position, size, range, etc. Therefore, the present disclosure is not limited to the position, size, range, etc. disclosed in the drawings and the like. DETAILED DESCRIPTION

[0031] Various exemplary embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. It should be noted that unless otherwise specifically stated, the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure.

[0032] The following description of at least one exemplary embodiment is in fact merely illustrative and is in no way intended to limit the present disclosure and its application or use. That is, the structures and methods herein are shown in an exemplary manner to illustrate different embodiments of the structures and methods in the present disclosure. However, those skilled in the art will appreciate that they merely illustrate exemplary ways of the present disclosure that can be implemented, rather than exhaustive ways. In addition, the drawings need not be drawn to scale, and some features may be enlarged to illustrate the details of specific components.

[0033] Technologies, methods, and apparatus known to ordinary technicians in the relevant field may not be discussed in detail, but where appropriate, such technologies, methods, and apparatus should be considered part of the authorization specification.

[0034] In order to achieve image recognition and classification, the following method is generally used in this field: using training images to train recognition models through object recognition technology. The specific training method includes collecting / developing data, preprocessing data, building models and training and evaluating them, debugging model hyperparameters, etc. After finally obtaining a model that meets the design goals, the model is used to recognize objects in new images.

[0035] Figure 1The processing flow of the training method according to the prior art is shown. First, the product is imaged as a whole to obtain an original image for data processing. In response to the original image received from the image acquisition system, the features to be extracted are manually determined and the type labels are manually marked. The extracted features include parameters related to the parts to be detected in the product, such as the model, position, color, size, etc. of the parts. The type label can be a selection box marked on the original image. It takes an average of 2 seconds to manually mark an image, and the amount of data required for model training requires at least 10,000 images to be marked. The original image and the type label are used as input and the extracted features are used as output to obtain training data to train image recognition models such as the YOLO model, optimize model parameters, and improve the generalization ability of the model, thereby generating a recognition model based on the parts to be detected. The object recognition method based on the model in the field includes: after image acquisition on the product production line, the original image obtained is manually marked, and the image marked with the type label is input into the recognition model for object recognition, and finally the features of the parts to be detected are obtained.

[0036] In contrast, according to the embodiments of the present disclosure, information obtained from the production system is used to automatically collect samples for different vehicle models and parts, without the need for manual labeling, which is time-consuming and relatively inaccurate, thereby shortening the training time of the model and improving work efficiency.

[0037] Figure 2 A schematic diagram of the processing flow of the training image classification model according to an embodiment of the present disclosure is shown. The imaging device 100 performs image acquisition from multiple angles on the product assembly line to obtain one or more product images 102. The production system 200 is a general term for systems including parts manufacturing, product assembly, etc., and is combined with the training and application of the image recognition model in the present disclosure. Information related to the product and its parts is received from the production system 200, and feature information of the part to be detected, such as the model of the part and the position in the product, is determined based on the relevant data obtained from the information, and feature extraction boxes 202 and 204 on the product image 102 are obtained. Since the information of the production system does not need to be manually marked, the corresponding part images 122 and 142 can be automatically identified from the overall product image based on the feature extraction box 202 or 204. Therefore, it is no longer necessary to use the complete product image 102 directly for the recognition model, and only the part images 122 and 142 as part of the product image 102 can be used as training data.

[0038] Next, the training process of the image classification model according to the present disclosure will be described. First, Figure 3 It shows the configuration of each device in the training system and the direction of data flow.

[0039] The imaging device 3100 may include an imaging unit, an A / D conversion unit, and an interface. The imaging unit has a configuration including an imaging optical system such as a lens and an image sensor, wherein the imaging optical system includes a mechanism for driving the lens, and the image sensor is a charge coupled device (CCD), a complementary metal oxide semiconductor (CMOS), etc. The image sensor photoelectrically converts light incident through the imaging optical system into an amount of charge to generate an image. The A / D conversion unit converts the output of the image sensor into a digital signal, and the interface provides a connection between the imaging device 3100 and an external device, and the captured image can be output to a subsequent processing unit via the interface.

[0040] The trigger for the imaging device 3100 to start acquiring images may be a predetermined input of the imaging device 3100, or may be a trigger condition reached by the production system 3200 when the product arrives at a specific position of the production line. During the process of assembling or transferring the product on the production line, the trigger condition includes at least one of the product arriving at a predetermined position on the production line and the assembly ratio of the product reaching a predetermined threshold. In some embodiments, the predetermined position may be the imaging range boundary of the imaging device 3100. When the product meets the trigger condition, the imaging device 3100 starts the image capture function and obtains the product image 3120.

[0041] The functions of the production system 3200 include the manufacture, transportation, assembly and inspection of parts of the product, and various parameters of the parts are collected during these manufacture, transportation, assembly and inspection processes. Different links of the production system 3200 may have a memory for storing part parameters to record data obtained when monitoring the production and manufacturing of parts. Examples of memories include magnetic storage devices such as hard disk drives (HDDs), semiconductor storage devices, optical storage devices, magneto-optical storage devices, etc. The production system 3200 may also include a data processor coupled to the memory to send the part parameters in the memory as information to other devices.

[0042] The information received from the production system 3200 is used to determine the first parameter 3220 and the second parameter 3240. The first parameter 3220 represents the model of the product on the production line, which is associated with the name, size, parts composition and other parameters of the product. For example, for the products of the automobile production line, according to its specific model, the size of the car body in views in different directions such as the front view, top view, left view, and bottom view, or the spatial data of the entire car in the coordinate system set according to the production line direction can be obtained. The parts composition corresponding to the model includes the type, quantity and installation position of the parts in each part according to different parts such as engine parts, transmission system parts, brake system parts, steering system parts, running system parts, and electrical instrument system parts.

[0043] The second parameter 3240 indicates the model of the part to be detected in the product, which is associated with parameters such as the subdivision type and appearance characteristics of the part. For example, if the part to be detected is a tire or a rearview mirror of a vehicle, its appearance characteristics include the size, color, etc. in views from different directions.

[0044] It can be seen that by combining the first parameter 3220 and the second parameter 3240, specific information such as the position of the part to be detected in the overall product can be determined, thereby extracting a sample image 3122 as an input of training data from the product image 3120.

[0045] Specifically, refer to Figure 4 A flowchart describing a training method for an image classification model according to the present disclosure. First, the imaging device 3100 performs an imaging operation at step S401 to obtain an overall product image 3120. The imaging device 3100 may have multiple imaging units installed at different angles, so the product image 3120 has views in different directions accordingly. At step S402, information associated with the product and its parts is received from the production system 3200 to determine a first parameter 3220 representing the product model and a second parameter 3240 representing the part model. Note that steps S401 and S402 are not necessarily performed sequentially. In some embodiments, S402 may be performed before S401, or the two steps may be performed in parallel.

[0046] In response to acquiring the product image 3120 and the first parameter 3220 and the second parameter 3240, the step proceeds to S403, the position of the part to be inspected is determined in the product image 3120 according to the first parameter 3220 and the second parameter 3240, and the sample image 3122 is automatically cut out from the product image 3120 according to the position. Corresponding to the number of product images 3120 and the product orientation in which they are presented, the sample image 3122 may have one or more viewing angles to comprehensively show the features of a single part to be inspected.

[0047] Subsequently, the processing step proceeds to S404, where the sample image 3122 and the second parameter 3240 are formed into training data in a one-to-one correspondence, which may be in the form of a vector of [sample image 3122, second parameter 3240]. In some embodiments, preferably, the sample image 3122 and the second parameter 3240 are stored in association in a memory. Such training data is input into the established classification model to perform part-based model training on it. The classification model is such as a MobileNet model. Finally, in step S405, an image recognition model 3400 according to the training method disclosed in the present invention is obtained.

[0048] In some embodiments, further, image classification models are trained for different parts in different products. Since the types of parts to be detected can represent parts with different functions and positions in automobile assembly, or parts with similar functions and positions that are distinguished according to product models, overall classification model training can be performed for all parts of the same product model, or model training can be performed using training data for similar parts.

[0049] As a non-limiting example, in the classification of automobile parts, sample images of rearview mirrors and wheels are used to train the image classification model. Through integration with the production system and imaging system, a large number of part images can be obtained to continuously iterate and generate image data. Compared with the recognition of the overall image and its specific position in the prior art, the image classification model here only contains the relationship between the product model, sample image and part model, and does not involve the parameter of position, which can reduce the number of network layers of the model and improve the training speed of the model.

[0050] By comparing the target recognition model based on the YOLO model in the prior art with the image classification model based on the MobileNet model according to the present disclosure, the following simulation results can be obtained:

[0051] Model Batch size Training time CPU Performance Model size Target recognition model 64 15 days 5000ms 200M+ Image Classification Model 64 1 day 30ms 20M

[0052] Wherein batch size (batch_size) refers to the number of images that enter the model training each time, and CPU performance refers to the execution time on the CPU to infer or identify an image. In the actual training of the present disclosure, the training time of the image classification model was shortened from 15 days to 1 day, the CPU performance only required 30ms, and because the size of the sample image was significantly reduced compared to the overall image, the final model size was also more streamlined. After simulation, the accuracy of the model obtained by the embodiment of the present disclosure can reach 98%. Therefore, the present disclosure can obtain a model that meets the usage conditions in a relatively short time.

[0053] The following will be combined Figure 5 and Figure 6 Describe the specific process of classifying or identifying product parts based on an image classification model.

[0054] Figure 5 FIG. 5 shows a schematic diagram of an image recognition system according to an embodiment of the present disclosure. Figure 3 The imaging device 3100 and the production system 3200 correspond to each other, and their detailed description is omitted here.

[0055] Information received from production system 5200 is used to determine first parameter 5220 and second parameter 5240. Figure 3 The first parameter 3220 in the image processing module 5220 indicates the model of the product on the production line, which is associated with the name, size, and component composition of the product. In the image recognition process, the model indicated by the first parameter 5220 can indicate the installation position of the part to be detected in the entire product. The second parameter 5240 indicates the model parameter of the part stored in the production system.

[0056] Further, in Figure 6 , a flowchart of an image recognition method according to an embodiment of the present disclosure is shown in FIG. 5 . The imaging device 5100 performs an imaging operation at step S601 to obtain an overall product image 5120. At step S602, information associated with a product and its parts is received from a production system 5200 to determine a first parameter 5220 representing a product model. Figure 4 Similarly, steps S601 and S602 may be performed in any order or in parallel.

[0057] In response to acquiring the product image 5120 and the first parameter 5220 , the step proceeds to S603 , where the position of the part to be detected is determined in the product image 5120 according to the first parameter 5220 , and a sample image 5122 is automatically cut out from the product image 5120 according to the position.

[0058] Then, the processing step proceeds to S604, the sample image 5122 is input into the image classification model 5400 for operation, and then the parameters of the part to be detected 5420 are output in S605. In some embodiments, further, after obtaining the output of the image classification model 5400, the part parameters 5420 are compared with the second parameters 5240 determined according to the information received from the production system 5200 to determine whether the part to be detected installed in the product is the correct model.

[0059] In some embodiments, the image classification model 5400 may be based on the aforementioned Figure 3 and Figure 4 The image classification model obtained by the training method in the illustrated embodiment, for example, the image classification model 3400.

[0060] The training method of the aforementioned image classification model and the processing of the image recognition method can be performed by hardware or by software. In the case of performing these processes by software, the program configuring the software is installed from the recording medium on a computing device embedded in dedicated hardware to run, or, for example, installed on a general computing device that can perform various functions by installing various programs.

[0061] Figure 7An exemplary configuration of a computing device that can implement an embodiment of the present disclosure is shown. The computing device includes one or more processors 1001, an input / output interface 1005 connected to the processor 1001 via a bus 1004, and memories 1002 and 1003 connected to the bus 1004. In some embodiments, the memory 1002 may be a read-only memory (ROM), and the memory 1003 may be a random access memory (RAM).

[0062] Processor 1001 may be any kind of processor, and may include, but is not limited to, one or more general-purpose processors or special-purpose processors (such as special-purpose processing chips). Memories 1002 and 1003 may be any non-transitory storage device that can implement data storage, and may include, but is not limited to, disk drives, optical storage devices, solid-state memories, floppy disks, flexible disks, hard disks, tapes or any other magnetic media, compressed disks or any other optical media, cache memories and / or any other storage chips or modules, and / or any other media from which a computer can read data, instructions and / or codes.

[0063] The bus 1004 may include, but is not limited to, an Industry Standard Architecture (ISA) bus, a Micro Channel Architecture (MCA) bus, an Enhanced ISA (EISA) bus, a Video Electronics Standards Association (VESA) local bus, and a Peripheral Component Interconnect (PCI) bus.

[0064] In some embodiments, the input / output interface 1005 is connected to an input unit 1006 configured by an input device such as a keyboard and a mouse for a user to input an operation command, an output unit 1007 that outputs a processing operation screen and an image of a processing result to a display device, a storage unit 1008 including a hard disk drive and the like for storing programs and various data, and a communication unit 1009 including a local area network (LAN) adapter and the like and performing communication processing via a network represented by the Internet. In addition, a drive 1010 that reads and writes data from and on a removable storage medium 1011 is connected.

[0065] Various aspects, embodiments, implementations or features of the foregoing embodiments may be used alone or in any combination. Various aspects of the foregoing embodiments may be implemented by software, hardware or a combination of hardware and software.

[0066] For example, the foregoing embodiments may be embodied as computer readable code on a computer readable medium. A computer readable medium is any data storage device that can store data that can thereafter be read by a computer system. Examples of computer readable media include read-only memory, random access memory, CD-ROM, DVD, magnetic tape, hard drive, solid state drive, and optical data storage device. The computer readable medium may also be distributed in a network coupled computer system so that the computer readable code is stored and executed in a distributed manner.

[0067] For example, the aforementioned embodiments may be in the form of hardware circuits. The hardware circuits may include any combination of combinational logic circuits, clock storage devices (such as floppy disks, flip-flops, latches, etc.), finite state machines, memories such as static random access memories or embedded dynamic random access memories, custom designed circuits, programmable logic arrays, etc.

[0068] In one embodiment, hardware circuits according to the present disclosure may be implemented by coding and designing one or more integrated circuits using a hardware description language (HDL) such as Verilog or VHDL, or in combination using discrete circuits.

[0069] In summary, the embodiments of the present disclosure may include the following configurations:

[0070] (1) A training method for an image classification model, comprising:

[0071] Get images of the product;

[0072] receiving information from a production system of a product to determine a first parameter representing a model of the product and a second parameter representing a model of a part of the product;

[0073] Determining a sample image corresponding to the part from the image of the product according to the first parameter and the second parameter; and

[0074] The sample image and the second parameter are input into the image classification model as training data to obtain an image classification model for identifying the part model of the product.

[0075] (2) The training method according to (1), wherein

[0076] The sample image is a portion of the image of the product.

[0077] (3) The training method according to (1), wherein

[0078] Based on the first parameter and the second parameter, a position of the sample image corresponding to the part on the image of the product is obtained.

[0079] (4) The training method according to (3), wherein

[0080] The image of the product is automatically cut based on the position to obtain a sample image.

[0081] (5) The training method according to any one of (1) to (4), wherein

[0082] The sample image is stored in memory in association with the second parameter.

[0083] (6) The training method according to any one of (1) to (5), wherein

[0084] The imaging device is triggered by the production system to capture an image of the product.

[0085] (7) The training method according to (6), wherein

[0086] The image of the product is captured in response to a trigger condition for image capture, the trigger condition comprising at least one of the product reaching a predetermined position on the production line and the assembly ratio of the product reaching a predetermined threshold.

[0087] (8) The training method according to any one of (1) to (7), wherein

[0088] For different categories of parts of the product, corresponding image classification models are trained respectively.

[0089] (9) An image recognition method using an image classification model, comprising:

[0090] Get images of the product;

[0091] receiving information from a production system of the product to determine a first parameter representing a model of the product;

[0092] Determining an image of a part to be inspected from an image of the product according to a first parameter;

[0093] The image of the part to be detected is input into the image classification model to determine the model of the part to be detected according to the image classification model.

[0094] (10) The image recognition method according to (9), wherein

[0095] The image classification model is an image classification model obtained according to the training method described in any one of (1) to (8).

[0096] (11) The image recognition method according to (9) or (10), wherein

[0097] The position of the part to be detected on the image of the product is obtained based on the first parameter.

[0098] (12) The image recognition method according to (11), wherein

[0099] The image of the product is automatically cut based on the position to obtain an image of the part to be inspected.

[0100] (13) The image recognition method according to any one of (9) to (12), further comprising:

[0101] receiving information from a product production system to determine a second parameter representing a model of a part to be inspected of the product; and

[0102] The second parameter is compared with the model of the part to be detected determined according to the image classification model to determine whether the model of the part to be detected installed on the product is correct.

[0103] (14) The image recognition method according to any one of (9) to (13), wherein

[0104] The imaging device is triggered by the production system to capture an image of the product.

[0105] (15) The image recognition method according to (14), wherein

[0106] The image of the product is captured in response to a trigger condition for image capture, the trigger condition comprising at least one of the product reaching a predetermined position on the production line and the assembly ratio of the product reaching a predetermined threshold.

[0107] (16) A computing device comprising:

[0108] a memory having instructions stored thereon; and

[0109] A processor configured to execute instructions to perform the training method according to any one of (1) to (8).

[0110] (17) A computing device comprising:

[0111] a memory having instructions stored thereon; and

[0112] A processor configured to execute the instructions to perform the image recognition method according to any one of (9) to (15).

[0113] (18) An image classification training system comprising:

[0114] an imaging device configured to capture an image of the product in response to a signal from the production system; and

[0115] A computing device according to (16).

[0116] (19) An image recognition system comprising:

[0117] an imaging device configured to capture an image of the product in response to a signal from the production system; and

[0118] A computing device according to (17).

[0119] (20) A computer-readable storage medium having instructions stored thereon, which, when executed by a computer, cause the computer to execute the training method described in any one of (1) to (8) or the image recognition method described in any one of (9) to (15).

[0120] Although some specific embodiments of the present invention have been shown in detail by example, it will be appreciated by those skilled in the art that the above examples are intended to be illustrative only and do not limit the scope of the present invention. It should be appreciated that some steps in the aforementioned method are not necessarily performed in the order shown in the figure, but they can be performed simultaneously, in different orders or in an overlapping manner. In addition, those skilled in the art can add some steps or omit some steps as needed. Some components in the aforementioned system are not necessarily arranged according to the figure, and those skilled in the art can add some components or omit some components as needed. It will be appreciated by those skilled in the art that the above embodiments can be modified without departing from the scope and essence of the present invention. The scope of the present invention is defined by the appended claims.

Claims

1. A training method for an image classification model, comprising: Get images of the product; Receiving information from a production system of a product to determine a first parameter representing a model of the product and a second parameter representing a model of a part of the product, wherein the first parameter is associated with a name, size, and component composition of the product, and wherein the second parameter is associated with a subdivision category and appearance feature of the part; Based on the first parameter and the second parameter, obtaining a position of a sample image corresponding to the part on the image of the product; Automatically cutting the image of the product based on the position to obtain the sample image; as well as The sample image and the second parameter are input as training data into an image classification model to obtain an image classification model for identifying a part model of a product.

2. The training method according to claim 1, wherein The sample image is stored in a memory in association with the second parameter.

3. The training method according to claim 1, wherein An imaging device is triggered by a production system to capture an image of the product.

4. The training method according to claim 3, wherein The image of the product is captured in response to a trigger condition for image capture, wherein the trigger condition includes at least one of the product reaching a predetermined position on the production line and the assembly ratio of the product reaching a predetermined threshold.

5. The training method according to claim 1, wherein For different categories of parts of the product, corresponding image classification models are trained respectively.

6. An image recognition method using an image classification model, comprising: Get images of the product; Receiving information from a production system of a product to determine a first parameter representing a model of the product, wherein the first parameter is associated with a name, size, and component composition of the product; Obtaining a position of the part to be detected on the image of the product based on the first parameter; Automatically cutting the image of the product based on the position to obtain an image of the part to be inspected; The image of the part to be detected is input into an image classification model to determine the model of the part to be detected according to the image classification model, wherein the image classification model is an image classification model obtained according to the training method described in any one of claims 1 to 5.

7. The image recognition method according to claim 6, further comprising: receiving information from a product production system to determine a second parameter representing a model of the part to be inspected of the product; as well as The second parameter is compared with the model of the part to be detected determined according to the image classification model to determine whether the model of the part to be detected installed on the product is correct.

8. The image recognition method according to claim 6, wherein An imaging device is triggered by the production system to capture an image of the product.

9. The image recognition method according to claim 8, wherein The image of the product is captured in response to a trigger condition for image capture, wherein the trigger condition includes at least one of the product reaching a predetermined position on the production line and the assembly ratio of the product reaching a predetermined threshold.

10. A computing device comprising: a memory having instructions stored therein; as well as A processor is configured to run the instructions to perform the training method according to any one of claims 1 to 5.

11. A computing device comprising: a memory having instructions stored therein; as well as A processor configured to run the instructions to perform the image recognition method according to any one of claims 6 to 9.

12. An image classification training system, comprising: an imaging device configured to capture an image of the product in response to a signal from the production system; as well as The computing device of claim 10.

13. An image recognition system, comprising: an imaging device configured to capture an image of the product in response to a signal from the production system; as well as The computing device of claim 11.

14. A computer-readable storage medium having instructions stored thereon, which, when executed by a computer, enables the computer to execute the training method according to any one of claims 1 to 5 or the image recognition method according to any one of claims 6 to 9.

Citation Information

Patent Citations

  • Component-based fine vehicle model recognition method

    CN106384100A

  • Recognition device for automobile parts based on deep convolutional network

    CN106980846A

  • Image-based vehicle maintenance plan

    US20180293552A1