Jewelry auxiliary detection method and device based on LoRA fine-tuning large model, computer equipment and readable storage medium

Abstract

The embodiment of the invention relates to the technical field of jewelry detection, and discloses a jewelry auxiliary detection method based on a LoRA fine-tuning large model, and the method comprises the steps: obtaining a to-be-detected image of to-be-detected jewelry; inputting a to-be-detected image into the fine-tuned detection large model to obtain a detection result; the detection result comprises a style, a carving type and a color corresponding to the jewelry to be detected; the fine-tuned detection large model is obtained by performing fine tuning on the large model by adopting a fine tuning technology based on a standardized jewelry sample; the standardized jewelry sample comprises a jewelry sample image and a standardized sample label; the standardized jewelry sample image comprises a jewelry sample image obtained after a camera device in a darkroom environment is adopted to shoot sample jewelry; the standardized sample marking comprises description of styles, carving types and colors; and after the detection result is analyzed, filling the style, carved shape and color of the jewelry to be detected into corresponding positions in the laboratory management system. According to the invention, high-accuracy detection and identification of multiple categories of jewelry are realized.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of jewelry detection and information technology, and specifically to a jewelry auxiliary detection method, device, computer equipment and readable storage medium based on a LoRA fine-tuning large model. Background Art

[0002] At present, algorithms for identifying objects in images include target detection algorithms, image classification algorithms, large model algorithms, etc. A series of attempts have been made by predecessors, for example, seawater pearl detection and recognition technology that integrates information entropy and feature extraction (Xu Ying, 2013), jewelry image recognition is mainly used in jewelry classification methods and similar product query methods for images (Ma Xinjie et al., 2018), methods for identifying jewelry quality based on images (Tao Zhendong et al., 2021; Zhang Xiaochen et al., 2022), a jewelry online monitoring method based on image processing (Shi Luchuan et al., 2023), and a gemstone cut recognition system in jewelry (Ma Ying et al., 2023), and jewelry quality detection methods and systems based on image recognition (Liu Lihong et al., 2024). Most of the above methods and technologies use target detection and deep learning image classification technology, but they are all detection and recognition of single categories, and the accuracy of methods for multiple recognition is relatively low.

[0003] In the existing technology, the description of jewelry style, cut, color, etc. requires a lot of manpower to identify and fill in, with an error rate of about 5%. However, in the face of tens of thousands of jewelry identification pictures every day, automatic image recognition technology is urgently needed to reduce human errors. Summary of the invention

[0004] In view of the above problems, the embodiments of the present invention provide a jewelry auxiliary detection method, device, computer equipment and readable storage medium based on the LoRA fine-tuning large model, which are used to solve the problems of low accuracy and low detection efficiency of multiple jewelry detections in the prior art.

[0005] According to one aspect of an embodiment of the present invention, a jewelry auxiliary detection method based on a LoRA fine-tuned large model is provided, the method comprising: Acquire an image of the jewelry to be detected; the image to be detected is obtained by photographing the jewelry to be detected in the darkroom environment using a camera device arranged in the darkroom environment; The image to be detected is input into the fine-tuned detection model to obtain a detection result; the detection result includes the style, cut and color of the jewelry to be detected; the fine-tuned detection model is obtained by fine-tuning the large model based on the standardized jewelry sample using the fine-tuning technology; the standardized jewelry sample includes a jewelry sample image and a standardized sample annotation; the standardized jewelry sample image includes a jewelry sample image obtained by photographing the sample jewelry using a camera device in a darkroom environment; the standardized sample annotation includes a description of the style, cut and color; After analyzing the test results, the style, cut and color of the jewelry to be tested are filled in the corresponding position in the laboratory management system.

[0006] In an optional manner, the step of obtaining the image of the jewelry to be inspected includes: The darkroom environment is formed by building a box body with baffles; the baffles are plates with an absorbance less than a preset absorbance threshold and no specular reflection; a jewelry fixing fixture and an industrial camera are arranged in the box body, a ring-shaped LED light source is arranged around the industrial camera, a layer of light-transmitting film is arranged between the LED light source and the inside of the box body, the absorbance of the light-transmitting film is less than the preset absorbance threshold, a polarized lens is arranged in front of the lens of the industrial camera, and at least one push-pull sealing structure is also arranged on the box body; The jewelry to be tested is placed on the jewelry fixing fixture, and the push-pull sealing structure is closed to seal the box; The industrial camera is controlled to shoot the jewelry to be inspected to obtain the image to be inspected.

[0007] In an optional manner, the industrial camera includes a plurality of cameras, each of which corresponds to a shooting angle; and controlling the industrial camera to shoot the jewelry to be inspected to obtain the image to be inspected includes: The cameras at various shooting angles are controlled to shoot the jewelry to be detected, and multiple initial images are obtained as the images to be detected.

[0008] In an optional manner, the fine-tuning technology is a LoRA structural algorithm; before inputting the image to be detected into a preset detection large model and obtaining the detection result, the method includes: Acquire multiple jewelry sample images taken at different angles of the jewelry sample in a darkroom environment; Using a JSON format file to describe each jewelry sample image in language, the language description includes a description of style, cut and color, and obtaining a standardized sample annotation for each jewelry sample image; using the jewelry sample image and the standardized sample annotation as a standardized jewelry sample; The standardized jewelry samples are input into the large model with the LoRA structural algorithm added for training, and blind sample reasoning verification is performed to obtain a fine-tuned large detection model.

[0009] In an optional manner, the standardized jewelry sample is input into a large model with a LoRA structure algorithm added thereto for training to obtain a fine-tuned detection large model, including: Input the standardized jewelry sample into the large model with the LoRA structure algorithm added to obtain the prediction result; Calculate the loss function based on the prediction results and standardized sample annotations; Adjusting parameters of the LoRA structure algorithm according to the loss function; Continue to input the standardized jewelry sample into the large model with the LoRA structural algorithm added to obtain the prediction result, calculate the loss function according to the prediction result and the standardized sample annotation, and adjust the parameters of the LoRA structural algorithm according to the loss function until the loss function is minimized to obtain the fine-tuned detection large model.

[0010] In an optional manner, the standardized jewelry sample is input into a large model with the LoRA structure algorithm added for training, and blind sample reasoning verification is performed to obtain a fine-tuned detection large model, including: Selecting a standardized jewelry sample formed by jewelry tested in history from the standardized jewelry sample as a first data set; Inputting the first data set into a large model with the LoRA structure algorithm added for training; From the remaining standardized jewelry samples, a first preset number of standardized jewelry samples are selected as blind samples for testing, and standardized jewelry samples with errors in the test are used as a second data set; Inputting the second data set into the large model with the LoRA structure algorithm added for training; From the remaining standardized main samples, a second preset number of standardized jewelry samples are selected as blind samples for testing, and the standardized jewelry samples that have tested errors are used as a third data set; Continue to iterate the training of the above steps until the prediction accuracy of the blind sample for style, cut and color is greater than a preset accuracy threshold, and then stop the training.

[0011] In an optional manner, after obtaining the image to be inspected of the jewelry to be inspected, the method further includes: sending the image number of the jewelry to be inspected and the image to be inspected to a laboratory management system; After analyzing the test results, the style, cut and color of the jewelry to be tested are filled in the corresponding position in the laboratory management system, including: Determine the image number of the jewelry to be tested corresponding to the test result; Determine the corresponding image number in the laboratory management system according to the image number; The style, cut and color of the jewelry to be tested are filled into the corresponding position of the image number of the laboratory management system through the API interface.

[0012] According to another aspect of an embodiment of the present invention, a jewelry auxiliary detection device based on a LoRA fine-tuning large model is provided, comprising: An acquisition module is used to acquire an image of the jewelry to be detected; the image to be detected is obtained by photographing the jewelry to be detected placed in the darkroom environment using a camera device set in the darkroom environment; A detection module, used for inputting the image to be detected into the fine-tuned detection model to obtain a detection result; the detection result includes the style, cut and color corresponding to the jewelry to be detected; the fine-tuned detection model is obtained by fine-tuning the large model based on a standardized jewelry sample using a fine-tuning technology; the standardized jewelry sample includes a jewelry sample image and a standardized sample annotation; the standardized jewelry sample image includes a jewelry sample image obtained by photographing the sample jewelry using a camera device in a darkroom environment; the standardized sample annotation includes a description of the style, cut and color; The analysis and filling module is used to fill in the style, cut and color of the jewelry to be tested into the corresponding position in the laboratory management system after analyzing the test results.

[0013] According to another aspect of an embodiment of the present invention, there is provided a computer device, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute the operation of the jewelry assisted detection method based on the LoRA fine-tuning large model.

[0014] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided, wherein the storage medium stores at least one executable instruction, and when the executable instruction is executed on a computer device, the computer device executes the operation of the jewelry assisted detection method based on the LoRA fine-tuning large model.

[0015] The embodiment of the present invention obtains an image of the jewelry to be detected, inputs the image to be detected into a fine-tuned detection model, and obtains a detection result; the detection result includes the style, cut and color corresponding to the jewelry to be detected; the fine-tuned detection model is based on a standardized jewelry sample, and is obtained by fine-tuning the large model using a fine-tuning technology; the standardized jewelry sample includes a jewelry sample image and a standardized sample annotation; the standardized jewelry sample image includes a jewelry sample image obtained by photographing the sample jewelry using a camera device in a darkroom environment; the standardized sample annotation includes a description of the style, cut and color; after analyzing the detection result, the style, cut and color of the jewelry to be detected are filled in the corresponding position in the laboratory management system, so that multiple categories of jewelry can be detected and identified with high accuracy, and a high accuracy rate is achieved by uniformly and standardizedly annotating the samples and training the large model, and the samples can be uploaded to the LIMS system after identification, thereby improving processing efficiency.

[0016] The above description is only an overview of the technical solution of the embodiment of the present invention. In order to more clearly understand the technical means of the embodiment of the present invention, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the embodiment of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings are only used to illustrate the embodiments and are not to be considered as limiting the present invention. In addition, the same reference symbols are used to represent the same components throughout the accompanying drawings. In the accompanying drawings: Figure 1 A schematic flow chart of a jewelry-assisted detection method based on a LoRA fine-tuning large model provided in an embodiment of the present invention is shown; Figure 2 The structure diagram of the jewelry auxiliary detection device based on the LoRA fine-tuning large model provided by an embodiment of the present invention is shown; Figure 3 A schematic diagram of the structure of a computer device provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0018] Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein.

[0019] The inventors of this application have made a lot of attempts in the recognition of jewelry styles, cuts, and colors, mainly trying YOLO, SSD, Fast R-CNN, Faster R-CNN, RetinaNet, CenterNet, FCOS, CornerNet, Deformable Part Model (DPM), Single Shot Object Detector (SSOD), VGGNet, AlexNet, VGGNet, GoogleNet, ZFNet, ResNet, Inception and other algorithms. Given that the comprehensive accuracy of the three recognitions is not high (50-64%), they cannot be used on a large scale in the production line. Moreover, the descriptions of styles and cuts made by the above algorithms for each person are slightly different, which makes the model very easy to confuse, especially the descriptions of colors are quite different and cannot meet the diversified representation.

[0020] The inventor of this application tried to directly call the existing large models, such as Qianwen_PLUS, LLaMA3, VILA, etc. The output results of the large models were different, with the highest accuracy of about 75%. However, the results given by the large models were inconsistent each time, and the descriptions were inconsistent with the descriptions of daily jewelry detection, and they could not be used at all. In view of the above problems, this application provides a method for achieving high-precision jewelry detection based on LoRA fine-tuning large models.

[0021] Figure 1 The flowchart of the jewelry auxiliary detection method based on the LoRA fine-tuning large model provided by the embodiment of the present invention is shown, and the method is executed by a computer device. The computer device can be a computer, a terminal device, a cloud computing device, a distributed device, etc., and the embodiment of the present invention does not make specific restrictions. Figure 1 As shown, the method comprises the following steps: Step 110: Acquire the image of the jewelry to be inspected.

[0022] In the embodiment of the present invention, the jewelry to be detected can be rings, bracelets, bracelets, necklaces, necklaces, earrings, pendants, pendants, brooches, corsages, accessories, headdresses, gold products, silver products, watches, ornaments, kits, raw materials, chains, etc.

[0023] The image to be detected is obtained by photographing the jewelry to be detected in the darkroom environment according to a camera device arranged in the darkroom environment. In order to standardize the photographed image and avoid the reflection problem, the embodiment of the present invention forms the darkroom environment by building a box with baffles; the baffle is a plate with an absorbance less than a preset absorbance threshold and no specular reflection; a jewelry fixing fixture and an industrial camera are arranged in the box, a ring-shaped LED light source is arranged around the industrial camera, a light-transmitting film is arranged between the LED light source and the inside of the box, the absorbance of the light-transmitting film is less than the preset absorbance threshold, a polarized lens is arranged in front of the lens of the industrial camera, and at least one push-pull sealing structure is also arranged on the box. The preset absorbance threshold can be 0.5. The jewelry to be detected is arranged on the jewelry fixing fixture, and the push-pull sealing structure is closed to seal the box; the industrial camera is controlled to shoot the jewelry to be detected to obtain the image to be detected. Specifically, when building the jewelry photography environment, a darkroom environment is built with baffles, and the darkroom environment is a hexahedral box. The baffle should be a plate with an absorbance of less than 0.5 and no specular reflection; a push-pull sealing structure is provided on one side of the box, through which jewelry samples can be placed into the darkroom environment of the box and then sealed; a jewelry fixing fixture is provided at the center point of the bottom of the darkroom environment of the box; an industrial camera is provided on the top of the hexahedron, and the camera has a pixel greater than 29 million; a ring-shaped LED light source with a color temperature of 6000-6500k is provided around the camera, and a layer of film with a transmittance greater than 80% is provided between the light source and the inside of the box, and the absorbance of the film is less than 0.5; a CPL polarized lens is placed in front of the camera lens.

[0024] In one embodiment of the present invention, the image to be detected may be an image taken from a specific angle. For example, it may be an image taken from directly above the jewelry fixing fixture. In another embodiment of the present invention, in order to enhance the accuracy of detection, the camera at each shooting angle is controlled to shoot the jewelry to be detected, and multiple initial images are obtained as the image to be detected. In this embodiment of the present invention, the image number of the jewelry to be detected and the image to be detected are also sent to the laboratory management system, and the laboratory management system establishes the archive information of the jewelry to be detected according to the image number and the image to be detected.

[0025] Step 120: input the image to be detected into a preset detection model to obtain a detection result.

[0026] The fine-tuned detection large model is obtained by fine-tuning the large model based on standardized jewelry samples using fine-tuning technology; the standardized jewelry samples include jewelry sample images and standardized sample annotations; the standardized jewelry sample images include jewelry sample images obtained by photographing sample jewelry using a camera device in a darkroom environment; the standardized sample annotations include style labels, cut labels and color labels. The fine-tuning tool is the LoRA (Low-Rank Adaptation) structure algorithm.

[0027] In the embodiment of the present invention, before executing step 120, the following steps are further performed: Step 001: Acquire multiple jewelry sample images taken at different angles in a darkroom environment.

[0028] Specifically, more than 10 million pictures of different types of jewelry taken in the same manner as step 110 are selected from jewelry testing laboratories in Pingzhou, Shanghai, Shenzhen, Beijing, etc., and jewelry pictures of different styles, cuts, and colors are aggregated and screened to select more than 1 million images for training.

[0029] Step 002: Use a JSON format file to describe each of the jewelry sample images in language, wherein the language description includes a description of the style, cut and color, and obtain a standardized sample annotation for each of the jewelry sample images; use the jewelry sample images and the standardized sample annotations as standardized jewelry samples.

[0030] Specifically, the images in the dataset are annotated using json format files, i.e., language descriptions. The styles are: "ring", "bracelet", "bracelet", "necklace", "set", "ear ornaments", "pendant", "pendant", "brooch", "corsage", "ornaments", "head ornaments", "gold products", "silver products", "watches", "ornaments", "kits", "raw materials", and "chains". The cut shapes are: "round diamond", "teardrop", "rectangular", "oval", "irregular", "square", "rectangular", "olive", "square", "round", "heart-shaped", "spherical", "carving", "ring", "free", "waist drum", "teardrop facet", "rectangular facet", "oval facet", "circular facet", "heart-shaped facet", "olive facet", "teardrop arc", "rectangular arc", "oval arc", "circular arc", "olive arc", "cylindrical", "irregular arc", "rectangular facet", "rectangular arc". The colors are: "white", "purple", "orange-red", "orange-yellow", "orange", "pink", "pink-purple", "brown-red", "brown-yellow", "brown", "black", "red", "yellow", "gray", "gray-white", "gray-black", "blue-green", "blue", "green", "light blue", "light green", "light purple", "creamy white", "dark red", "yellow-green", "golden yellow". The unified language description can be, for example, "bracelet, ring, light green", "pendant, teardrop-shaped facet, purple-red", etc., and the language and text are used to limit the content extracted from the picture to "extract the type in the picture". Step 003: Input the standardized jewelry sample into the large model with the LoRA structure algorithm added for training to obtain the preset detection large model.

[0031] Wherein, step 003 specifically includes the following steps: Step 0031: Input the standardized jewelry sample into the large model with the LoRA structure algorithm added to obtain the prediction result.

[0032] Among them, in an embodiment of the present invention, when the image to be detected is only one image, the annotated image to be detected is directly input into the large model with the LoRA structure algorithm added for training to obtain a prediction result. When the image to be detected includes multiple initial images at different angles, the multiple initial images are first fused before inputting the model. Among them, the specific process of fusing multiple initial images includes: preprocessing each captured initial image, including resizing, normalization, cropping and other operations to ensure that they have the same size and format, which is convenient for subsequent processing and fusion. The initial images of each angle are weighted averaged according to the importance of different angles. For example, in one embodiment, the first initial image captured by the camera device placed at the top is used as the main image, which has the highest importance, and the second initial image captured by the camera device in the surrounding area is used as the auxiliary image, which has a lower importance than the first initial image, and the initial images captured by each camera include the first initial image and the second initial image. In one embodiment of the present invention, the image weight of the first initial image is set to 0.8, and the image weights of the remaining second initial images are set to 0.2. In another embodiment of the present invention, during annotation, the first initial image and each second initial image may be weighted averaged to obtain a fused image, and the fused image may be input into a large model to which the LoRA structure algorithm is added to obtain a prediction result. In yet another embodiment of the present invention, the image weights of each angle are obtained by adjusting during the training process.

[0033] Step 0032: Calculate the loss function based on the prediction results and the standardized sample annotations. In this embodiment of the present invention, a bypass is added next to the original pre-trained language model. n×m Without training, the optimization process is completed by adding the residual ∆W.

[0034] The optimization process is: (W+∆W recognition )x = Wx + ABx; Formula 1 Where W∈R d*d , ∆W recognition =AB, ∆W is the product of two low-rank matrices A and B. A∈R n×d , B∈R n×d Calculate the weight matrix d under certain parameters. This matrix is ​​much smaller than n. Select a suitable rank r to reduce the number of parameters in the large model.

[0035] L recognition = Σ (M_ij - (A × B)_ij)² Formula 2 Among them, L is the loss function, Σ represents the sum, M_ij is the element in the matrix M, and (A × B)_ij is the element at the corresponding position in the approximation matrix. The process of adjusting the loss function is to adjust the values ​​of A and B to minimize L.

[0036] Step 0033: Adjust the parameters of the LoRA structure algorithm according to the loss function, that is, adjust the values ​​of A and B.

[0037] Step 0034: Continue to input the standardized jewelry sample into the large model with the LoRA structural algorithm added to obtain the prediction result, calculate the loss function according to the prediction result and the standardized sample annotation, and adjust the parameters of the LoRA structural algorithm according to the loss function until the loss function is minimized to obtain the preset detection large model.

[0038] Among them, when the loss function is minimized, the final values ​​of parameters A and B are obtained, that is, the weights of the LoRA structural algorithm. The fine-tuned LoRA structural algorithm weights are merged with the general large model as the final fine-tuned detection large model constructed for model reasoning to achieve high-accuracy reasoning.

[0039] In the embodiment of the present invention, in order to minimize the impact of the large model's large parameters requiring large computing power, the embodiment of the present invention adopts a method of multiple training splits for training. Among them, the standardized jewelry samples formed by the historically detected jewelry are selected from the standardized jewelry samples as the first data set; the first data set is input into the large model with the LoRA structural algorithm added for training; from the remaining standardized jewelry samples, a first preset number of standardized jewelry samples are selected as blind samples for testing, and the standardized jewelry samples with errors are used as the second data set; the second data set is input into the large model with the LoRA structural algorithm added for training; from the remaining standardized main samples, a second preset number of standardized jewelry samples are selected as blind samples for testing, and the standardized jewelry samples with errors are used as the third data set; the iterative training of the above steps is continued until the prediction accuracy of the blind sample for style, cut and color is greater than the preset accuracy threshold, and the training is stopped. Specifically, in the first stage, data set 1 is used for training. Data set 1 needs to cover more than 30 styles, more than 30 cuts, and more than 60 colors encountered in front-line inspections. 2,000 selected blind samples are used for testing, and the ones with errors are selected and put into data set 1 to form data set 2. Data set 2 is trained, and data sets 3, 4, 5, 6, etc. are operated cyclically according to the above method until the comprehensive accuracy of the three items of the blind samples is greater than 90%, that is, (style recognition accuracy * cut recognition accuracy * color recognition accuracy) / 3 = comprehensive accuracy greater than 90%, so that the weights trained by this data set are used as the weights in the final algorithm.

[0040] Step 130: After analyzing the test results, fill in the style, cut and color of the jewelry to be tested into the corresponding position in the laboratory management system.

[0041] Among them, the embodiment of the present invention determines the image number of the jewelry to be tested corresponding to the test result; determines the corresponding image number in the laboratory management system according to the image number; and fills the style, cut and color of the jewelry to be tested into the corresponding position of the image number of the laboratory management system through the API interface. In this way, the detection and information upload of jewelry can be automatically realized online, replacing the manual filling method in the prior art, greatly improving production efficiency and reducing the error rate caused by filling, from the original 5-7% to about 1%. The use of information technology can realize the automatic recognition of front-line detection pictures and fill them into the existing management system, and can automatically process tens of thousands of images every day, which is much more efficient than manual input by personnel.

[0042] The embodiment of the present invention obtains the image to be detected of the jewelry to be detected, inputs the image to be detected into the fine-tuned detection model, and obtains the detection result; the detection result includes the style, cut and color corresponding to the jewelry to be detected; the fine-tuned detection model is based on the standardized jewelry sample, and the large model is fine-tuned by fine-tuning the large model by fine-tuning technology; the standardized jewelry sample includes a jewelry sample image and a standardized sample annotation; the standardized jewelry sample image includes a jewelry sample image obtained by shooting the sample jewelry with a camera device in a darkroom environment; the standardized sample annotation includes a description of the style, cut and color; after parsing the detection result, the style, cut and color of the jewelry to be detected are filled in the corresponding position in the laboratory management system, and multiple categories of jewelry can be detected and identified with high accuracy. By uniformly standardizing the samples and training the large model, a high accuracy rate is achieved, and after identification, it can be uploaded to the LIMS system, thereby improving processing efficiency. The embodiment of the present invention uses an industrial camera to shoot different jewelry for daily inspection, and tries to collect different jewelry and jade jewelry into a picture data set. The json format file is used to uniformly annotate the styles, cuts, and colors of the pictures in the dataset. A general large model is used, and the LoRA structure algorithm is used to fine-tune the Q, K, V, and O matrices of the model. The low-rank decomposition greatly reduces the number of parameters, making the original styles (recognition accuracy 90%), cuts (80-90%), and colors (recognition accuracy 70-80%). The comprehensive accuracy (three-item product 50-64%) has been greatly improved. The comprehensive accuracy of the three unified recognitions of this method is greater than 90%. Use python to write the corresponding code, and upload the images of different jewelry to the LIMS management system after recognition, so as to facilitate the rapid printing of paper certificates. It can solve the problem of image recognition and filling in more accurately and automatically.

[0043] Figure 2 FIG. 1 is a schematic diagram showing the structure of a jewelry auxiliary detection device based on a LoRA fine-tuning large model provided by an embodiment of the present invention. Figure 2 As shown, the device 300 includes: The acquisition module 310 is used to acquire an image of the jewelry to be detected; the image to be detected is obtained by photographing the jewelry to be detected placed in the darkroom environment using a camera device set in the darkroom environment; The detection module 320 is used to input the image to be detected into the fine-tuned detection model to obtain the detection result; the fine-tuned detection model is based on the standardized jewelry sample and is obtained by fine-tuning the large model using the fine-tuning technology; the standardized jewelry sample includes a jewelry sample image and a standardized sample annotation; the standardized jewelry sample image includes a jewelry sample image obtained by photographing the sample jewelry using a camera device in a darkroom environment; the standardized sample annotation includes a style label, a cut label and a color label; The analysis and filling module 330 is used to fill in the style, cut and color of the jewelry to be tested into the corresponding position in the laboratory management system after analyzing the test results.

[0044] In an optional manner, the step of obtaining the image of the jewelry to be inspected includes: The darkroom environment is formed by building a box body with baffles; the baffles are plates with an absorbance less than a preset absorbance threshold and no specular reflection; a jewelry fixing fixture and an industrial camera are arranged in the box body, a ring-shaped LED light source is arranged around the industrial camera, a layer of light-transmitting film is arranged between the LED light source and the inside of the box body, the absorbance of the light-transmitting film is less than the preset absorbance threshold, a polarized lens is arranged in front of the lens of the industrial camera, and at least one push-pull sealing structure is also arranged on the box body; The jewelry to be tested is placed on the jewelry fixing fixture, and the push-pull sealing structure is closed to seal the box; The industrial camera is controlled to shoot the jewelry to be inspected to obtain the image to be inspected.

[0045] In an optional manner, the industrial camera includes a plurality of cameras, each of which corresponds to a shooting angle; and controlling the industrial camera to shoot the jewelry to be inspected to obtain the image to be inspected includes: The cameras at various shooting angles are controlled to shoot the jewelry to be detected, and multiple initial images are obtained as the images to be detected.

[0046] In an optional manner, the fine-tuning tool is a LoRA structural algorithm; before inputting the image to be detected into the fine-tuned detection model to obtain the detection result, the device further includes: The sample image acquisition module is used to acquire multiple jewelry sample images taken at different angles of the jewelry sample in a darkroom environment; A standardized annotation module is used to describe each jewelry sample image in a language using a JSON format file, wherein the language description includes a description of style, cut and color, and obtain a standardized sample annotation for each jewelry sample image; and the jewelry sample image and the standardized sample annotation are used as a standardized jewelry sample; The training module is used to input the standardized jewelry samples into the large model with the LoRA structure algorithm added for training, and perform blind sample reasoning verification to obtain a fine-tuned detection large model.

[0047] In an optional manner, the standardized jewelry sample is input into a large model with a LoRA structure algorithm added thereto for training to obtain a fine-tuned large detection model, including: Input the standardized jewelry sample into the large model with the LoRA structure algorithm added to obtain the prediction result; Calculate the loss function based on the prediction results and standardized sample annotations; Adjusting the parameters of the LoRA structure algorithm according to the loss function; Continue to input the standardized jewelry sample into the large model with the LoRA structural algorithm added to obtain the prediction result, calculate the loss function according to the prediction result and the standardized sample annotation, and adjust the parameters of the LoRA structural algorithm according to the loss function until the loss function is minimized to obtain the fine-tuned detection large model.

[0048] In an optional manner, the standardized jewelry sample is input into a large model with the LoRA structure algorithm added for training, and blind sample reasoning verification is performed to obtain a fine-tuned detection large model, including: Selecting a standardized jewelry sample formed by jewelry tested in history from the standardized jewelry sample as a first data set; Inputting the first data set into a large model with the LoRA structure algorithm added for training; From the remaining standardized jewelry samples, a first preset number of standardized jewelry samples are selected as blind samples for testing, and standardized jewelry samples with errors in the test are used as a second data set; Inputting the second data set into the large model with the LoRA structure algorithm added for training; From the remaining standardized main samples, a second preset number of standardized jewelry samples are selected as blind samples for testing, and the standardized jewelry samples that have tested errors are used as a third data set; Continue to iterate the training of the above steps until the prediction accuracy of the blind sample for style, cut and color is greater than a preset accuracy threshold, and then stop the training.

[0049] In an optional manner, after acquiring the image of the jewelry to be detected, the device further includes: A sending module, used for sending the image number of the jewelry to be tested and the image to be tested to the laboratory management system; After analyzing the test results, the style, cut and color of the jewelry to be tested are filled in the corresponding position in the laboratory management system, including: Determine the image number of the jewelry to be tested corresponding to the test result; Determine the corresponding image number in the laboratory management system according to the image number; The style, cut and color of the jewelry to be tested are filled into the corresponding position of the image number of the laboratory management system through the API interface.

[0050] The embodiment of the present invention obtains an image of the jewelry to be detected, inputs the image to be detected into a fine-tuned detection model, and obtains a detection result; the detection result includes the style, cut and color corresponding to the jewelry to be detected; the fine-tuned detection model is based on a standardized jewelry sample, and is obtained by fine-tuning the large model using a fine-tuning tool; the standardized jewelry sample includes a jewelry sample image and a standardized sample annotation; the standardized jewelry sample image includes a jewelry sample image obtained by photographing the sample jewelry using a camera device in a darkroom environment; the standardized sample annotation includes a description of the style, cut and color; after analyzing the detection result, the style, cut and color of the jewelry to be detected are filled in the corresponding position in the laboratory management system, so that multiple categories of jewelry can be detected and identified with high accuracy, and a high accuracy rate is achieved by uniformly and standardizedly annotating the samples and training the large model. After identification, it can be uploaded to the LIMS system, thereby improving processing efficiency.

[0051] Figure 3 The schematic diagram of the structure of the computer device provided by the embodiment of the present invention is shown. The specific embodiment of the present invention does not limit the specific implementation of the computer device.

[0052] like Figure 3 As shown, the computer device may include: a processor (processor) 402 , a communication interface (Communications Interface) 404 , a memory (memory) 406 , and a communication bus 408 .

[0053] The processor 402, the communication interface 404, and the memory 406 communicate with each other via the communication bus 408. The communication interface 404 is used to communicate with other devices such as a client or other server network elements. The processor 402 is used to execute the program 410, which can specifically execute the relevant steps in the above-mentioned embodiment of the jewelry auxiliary detection method based on the LoRA fine-tuning large model.

[0054] Specifically, the program 410 may include program code including computer executable instructions.

[0055] The processor 402 may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present invention. The one or more processors included in the computer device may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0056] The memory 406 is used to store the program 410. The memory 406 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0057] The program 410 may be specifically called by the processor 402 to enable the computer device to perform the following operations: Acquire an image of the jewelry to be detected; the image to be detected is obtained by photographing the jewelry to be detected in the darkroom environment using a camera device arranged in the darkroom environment; The image to be detected is input into the fine-tuned detection model to obtain a detection result; the detection result includes the style, cut and color of the jewelry to be detected; the fine-tuned detection model is obtained by fine-tuning the large model based on the standardized jewelry sample using the fine-tuning technology; the standardized jewelry sample includes a jewelry sample image and a standardized sample annotation; the standardized jewelry sample image includes a jewelry sample image obtained by photographing the sample jewelry using a camera device in a darkroom environment; the standardized sample annotation includes a description of the style, cut and color; After analyzing the test results, the style, cut and color of the jewelry to be tested are filled in the corresponding position in the laboratory management system.

[0058] In an optional manner, the step of obtaining the image of the jewelry to be inspected includes: The darkroom environment is formed by building a box body with baffles; the baffles are plates with an absorbance less than a preset absorbance threshold and no specular reflection; a jewelry fixing fixture and an industrial camera are arranged in the box body, a ring-shaped LED light source is arranged around the industrial camera, a layer of light-transmitting film is arranged between the LED light source and the inside of the box body, the absorbance of the light-transmitting film is less than the preset absorbance threshold, a polarized lens is arranged in front of the lens of the industrial camera, and at least one push-pull sealing structure is also arranged on the box body; The jewelry to be tested is placed on the jewelry fixing fixture, and the push-pull sealing structure is closed to seal the box; The industrial camera is controlled to shoot the jewelry to be inspected to obtain the image to be inspected.

[0059] In an optional manner, the industrial camera includes a plurality of cameras, each of which corresponds to a shooting angle; and controlling the industrial camera to shoot the jewelry to be inspected to obtain the image to be inspected includes: The cameras at various shooting angles are controlled to shoot the jewelry to be detected, and multiple initial images are obtained as the images to be detected.

[0060] In an optional manner, the fine-tuning tool is a LoRA structural algorithm; before inputting the image to be detected into the fine-tuned detection model and obtaining the detection result, the method includes: Acquire multiple jewelry sample images taken at different angles of the jewelry sample in a darkroom environment; Using a JSON format file to describe each jewelry sample image in language, the language description includes a description of style, cut and color, and obtaining a standardized sample annotation for each jewelry sample image; using the jewelry sample image and the standardized sample annotation as a standardized jewelry sample; The standardized jewelry samples are input into the large model with the LoRA structure algorithm added for training to obtain a fine-tuned detection large model.

[0061] In an optional manner, the standardized jewelry sample is input into a large model with a LoRA structure algorithm added thereto for training to obtain a fine-tuned detection large model, including: Input the standardized jewelry sample into the large model with the LoRA structure algorithm added to obtain the prediction result; Calculate the loss function based on the prediction results and standardized sample annotations; Adjusting the parameters of the LoRA structure algorithm according to the loss function; Continue to input the standardized jewelry sample into the large model with the LoRA structural algorithm added to obtain the prediction result, calculate the loss function according to the prediction result and the standardized sample annotation, and adjust the parameters of the LoRA structural algorithm according to the loss function until the loss function is minimized to obtain the fine-tuned detection large model.

[0062] In an optional manner, the standardized jewelry sample is input into a large model with a LoRA structure algorithm added thereto for training to obtain a fine-tuned large detection model, including: Selecting a standardized jewelry sample formed by jewelry tested in history from the standardized jewelry sample as a first data set; Inputting the first data set into a large model with the LoRA structure algorithm added for training; From the remaining standardized jewelry samples, a first preset number of standardized jewelry samples are selected as blind samples for testing, and standardized jewelry samples with errors in the test are used as a second data set; Inputting the second data set into the large model with the LoRA structure algorithm added for training; From the remaining standardized main samples, a second preset number of standardized jewelry samples are selected as blind samples for testing, and the standardized jewelry samples that have tested errors are used as a third data set; Continue to iterate the training of the above steps until the prediction accuracy of the blind sample for style, cut and color is greater than a preset accuracy threshold, and then stop the training.

[0063] In an optional manner, after obtaining the image to be inspected of the jewelry to be inspected, the method further includes: sending the image number of the jewelry to be inspected and the image to be inspected to a laboratory management system; After analyzing the test results, the style, cut and color of the jewelry to be tested are filled in the corresponding position in the laboratory management system, including: Determine the image number of the jewelry to be tested corresponding to the test result; Determine the corresponding image number in the laboratory management system according to the image number; Fill in the style, cut and color of the jewelry to be tested into the corresponding position of the image number of the laboratory management system.

[0064] The embodiment of the present invention obtains an image of the jewelry to be detected, inputs the image to be detected into a fine-tuned detection model, and obtains a detection result; the detection result includes the style, cut and color corresponding to the jewelry to be detected; the fine-tuned detection model is based on a standardized jewelry sample, and is obtained by fine-tuning the large model using a fine-tuning technology; the standardized jewelry sample includes a jewelry sample image and a standardized sample annotation; the standardized jewelry sample image includes a jewelry sample image obtained by photographing the sample jewelry using a camera device in a darkroom environment; the standardized sample annotation includes a description of the style, cut and color; after analyzing the detection result, the style, cut and color of the jewelry to be detected are filled in the corresponding position in the laboratory management system, so that multiple categories of jewelry can be detected and identified with high accuracy, and a high accuracy rate is achieved by uniformly and standardizedly annotating the samples and training the large model, and the samples can be uploaded to the LIMS system after identification, thereby improving processing efficiency.

[0065] An embodiment of the present invention provides a computer-readable storage medium, which stores at least one executable instruction. When the executable instruction is executed on a computer device, the computer device executes the jewelry assisted detection method based on the LoRA fine-tuning large model in any of the above method embodiments.

[0066] The executable instructions can be specifically used to cause the computer device to perform the following operations: Acquire an image of the jewelry to be detected; the image to be detected is obtained by photographing the jewelry to be detected in the darkroom environment using a camera device arranged in the darkroom environment; The image to be detected is input into the fine-tuned detection model to obtain a detection result; the detection result includes the style, cut and color of the jewelry to be detected; the fine-tuned detection model is obtained by fine-tuning the large model based on the standardized jewelry sample using the fine-tuning technology; the standardized jewelry sample includes a jewelry sample image and a standardized sample annotation; the standardized jewelry sample image includes a jewelry sample image obtained by photographing the sample jewelry using a camera device in a darkroom environment; the standardized sample annotation includes a description of the style, cut and color; After analyzing the test results, the style, cut and color of the jewelry to be tested are filled in the corresponding position in the laboratory management system.

[0067] In an optional manner, the step of obtaining the image of the jewelry to be inspected includes: The darkroom environment is formed by building a box body with baffles; the baffles are plates with an absorbance less than a preset absorbance threshold and no specular reflection; a jewelry fixing fixture and an industrial camera are arranged in the box body, a ring-shaped LED light source is arranged around the industrial camera, a layer of light-transmitting film is arranged between the LED light source and the inside of the box body, the absorbance of the light-transmitting film is less than the preset absorbance threshold, a polarized lens is arranged in front of the lens of the industrial camera, and at least one push-pull sealing structure is also arranged on the box body; The jewelry to be tested is placed on the jewelry fixing fixture, and the push-pull sealing structure is closed to seal the box; The industrial camera is controlled to shoot the jewelry to be inspected to obtain the image to be inspected.

[0068] In an optional manner, the industrial camera includes a plurality of cameras, each of which corresponds to a shooting angle; and controlling the industrial camera to shoot the jewelry to be inspected to obtain the image to be inspected includes: The cameras at various shooting angles are controlled to shoot the jewelry to be detected, and multiple initial images are obtained as the images to be detected.

[0069] In an optional manner, the fine-tuning tool is a LoRA structural algorithm; before inputting the image to be detected into the fine-tuned detection model and obtaining the detection result, the method includes: Get images of jewelry samples taken in a darkroom environment; Using a JSON format file to describe each jewelry sample image in language, the language description includes a description of style, cut and color, and obtaining a standardized sample annotation for each jewelry sample image; using the jewelry sample image and the standardized sample annotation as a standardized jewelry sample; The standardized jewelry samples are input into the large model with the LoRA structure algorithm added for training to obtain a fine-tuned detection large model.

[0070] In an optional manner, the standardized jewelry sample is input into a large model with a LoRA structure algorithm added thereto for training to obtain a fine-tuned large detection model, including: Input the standardized jewelry sample into the large model with the LoRA structure algorithm added to obtain the prediction result; Calculate the loss function based on the prediction results and standardized sample annotations; Adjusting the parameters of the LoRA structure algorithm according to the loss function; Continue to input the standardized jewelry sample into the large model with the LoRA structural algorithm added to obtain the prediction result, calculate the loss function according to the prediction result and the standardized sample annotation, and adjust the parameters of the LoRA structural algorithm according to the loss function until the loss function is minimized to obtain the fine-tuned detection large model.

[0071] In an optional manner, the standardized jewelry sample is input into a large model with a LoRA structure algorithm added thereto for training to obtain a fine-tuned large detection model, including: Selecting a standardized jewelry sample formed by jewelry tested in history from the standardized jewelry sample as a first data set; Inputting the first data set into a large model with the LoRA structure algorithm added for training; From the remaining standardized jewelry samples, a first preset number of standardized jewelry samples are selected as blind samples for testing, and standardized jewelry samples with errors in the test are used as a second data set; Inputting the second data set into the large model with the LoRA structure algorithm added for training; From the remaining standardized main samples, a second preset number of standardized jewelry samples are selected as blind samples for testing, and the standardized jewelry samples that have tested errors are used as a third data set; Continue to iterate the training of the above steps until the prediction accuracy of the blind sample for style, cut and color is greater than a preset accuracy threshold, and then stop the training.

[0072] In an optional manner, after obtaining the image to be inspected of the jewelry to be inspected, the method further includes: sending the image number of the jewelry to be inspected and the image to be inspected to a laboratory management system; After analyzing the test results, the style, cut and color of the jewelry to be tested are filled in the corresponding position in the laboratory management system, including: Determine the image number of the jewelry to be tested corresponding to the test result; Determine the corresponding image number in the laboratory management system according to the image number; Fill in the style, cut and color of the jewelry to be tested into the corresponding position of the image number of the laboratory management system.

[0073] The embodiment of the present invention obtains an image of the jewelry to be detected, inputs the image to be detected into a fine-tuned detection model, and obtains a detection result; the detection result includes the style, cut and color corresponding to the jewelry to be detected; the fine-tuned detection model is based on a standardized jewelry sample, and is obtained by fine-tuning the large model using a fine-tuning tool; the standardized jewelry sample includes a jewelry sample image and a standardized sample annotation; the standardized jewelry sample image includes a jewelry sample image obtained by photographing the sample jewelry using a camera device in a darkroom environment; the standardized sample annotation includes a description of the style, cut and color; after analyzing the detection result, the style, cut and color of the jewelry to be detected are filled in the corresponding position in the laboratory management system, so that multiple categories of jewelry can be detected and identified with high accuracy, and a high accuracy rate is achieved by uniformly and standardizedly annotating the samples and training the large model. After identification, it can be uploaded to the LIMS system, thereby improving processing efficiency.

[0074] An embodiment of the present invention provides a jewelry assisted detection device based on a LoRA fine-tuning large model, which is used to execute the above-mentioned jewelry assisted detection method based on a LoRA fine-tuning large model.

[0075] An embodiment of the present invention provides a computer program, which can be called by a processor to enable a computer device to execute the jewelry assisted detection method based on the LoRA fine-tuning large model in any of the above method embodiments.

[0076] An embodiment of the present invention provides a computer program product, which includes a computer program stored on a computer-readable storage medium, and the computer program includes program instructions. When the program instructions are run on a computer, the computer executes the jewelry assisted detection method based on the LoRA fine-tuning large model in any of the above method embodiments.

[0077] The algorithm or display provided herein is not inherently related to any particular computer, virtual system or other equipment. Various general purpose systems can also be used together with the teachings based on this. According to the above description, it is obvious to construct the structure required for this type of system. In addition, the embodiment of the present invention is not directed to any specific programming language yet. It should be understood that various programming languages ​​can be utilized to realize the content of the present invention described herein, and the description made to specific languages ​​above is for disclosing the best mode of the present invention.

[0078] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. In some instances, well-known methods, structures and techniques are not shown in detail so as not to obscure the understanding of this description.

[0079] Similarly, it should be understood that in order to streamline the present invention and aid in understanding one or more of the various inventive aspects, in the above description of exemplary embodiments of the present invention, various features of the embodiments of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. However, this method of disclosure should not be interpreted as reflecting an intention that the claimed invention requires more features than those expressly recited in each claim.

[0080] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and set in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and may be divided into multiple submodules or subunits or subcomponents. All features disclosed in this specification (including the accompanying claims, abstracts and drawings) and all processes or units of any method or device disclosed in this manner may be combined in any combination, except that at least some of such features and / or processes or units are mutually exclusive. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstracts and drawings) may be replaced by an alternative feature that provides the same, equivalent or similar purpose.

[0081] It should be noted that the above embodiments illustrate the present invention rather than limit it, and that those skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets shall not be construed as a limitation on the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention may be implemented by means of hardware comprising a number of different elements and by means of a suitably programmed computer. In a unit claim enumerating a number of devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc. does not indicate any order. These words may be interpreted as names. The steps in the above embodiments, unless otherwise specified, should not be understood as limitations on the order of execution.

Claims

1. A jewelry-assisted detection method based on LoRA fine-tuning large model, characterized in that: The method comprises: Acquire an image of the jewelry to be detected; the image to be detected is obtained by photographing the jewelry to be detected in the darkroom environment using a camera device arranged in the darkroom environment; The image to be detected is input into the fine-tuned detection model to obtain a detection result; the detection result includes the style, cut and color of the jewelry to be detected; the fine-tuned detection model is obtained by fine-tuning the large model based on the standardized jewelry sample using the fine-tuning technology; the standardized jewelry sample includes a jewelry sample image and a standardized sample annotation; the standardized jewelry sample image includes a jewelry sample image obtained by photographing the sample jewelry using a camera device in a darkroom environment; the standardized sample annotation includes a description of the style, cut and color; After analyzing the test results, the style, cut and color of the jewelry to be tested are filled in the corresponding position in the laboratory management system.

2. The method according to claim 1, characterized in that: The step of obtaining the image of the jewelry to be detected includes: The darkroom environment is formed by building a box body with baffles; the baffles are plates with an absorbance less than a preset absorbance threshold and no specular reflection; a jewelry fixing fixture and an industrial camera are arranged in the box body, a ring-shaped LED light source is arranged around the industrial camera, a layer of light-transmitting film is arranged between the LED light source and the inside of the box body, the absorbance of the light-transmitting film is less than the preset absorbance threshold, a polarized lens is arranged in front of the lens of the industrial camera, and at least one push-pull sealing structure is also arranged on the box body; The jewelry to be tested is placed on the jewelry fixing fixture, and the push-pull sealing structure is closed to seal the box; The industrial camera is controlled to shoot the jewelry to be inspected to obtain the image to be inspected.

3. The method according to claim 2, characterized in that The industrial camera includes a plurality of cameras, each of which corresponds to a shooting angle; and controlling the industrial camera to shoot the jewelry to be inspected to obtain the image to be inspected includes: The cameras at various shooting angles are controlled to shoot the jewelry to be detected, and multiple initial images are obtained as the images to be detected.

4. The method according to any one of claims 1 to 3, characterized in that: The fine-tuning technology is the LoRA structure algorithm; before the image to be detected is input into the fine-tuned detection model and the detection result is obtained, the method includes: Acquire multiple jewelry sample images taken at different angles of the jewelry sample in a darkroom environment; Using a JSON format file to describe each jewelry sample image in language, the language description includes a description of style, cut and color, and obtaining a standardized sample annotation for each jewelry sample image; using the jewelry sample image and the standardized sample annotation as a standardized jewelry sample; The standardized jewelry samples are input into a large model with the LoRA structural algorithm added for training, and blind sample reasoning verification is performed to obtain a fine-tuned large detection model.

5. The method according to claim 4, characterized in that The standardized jewelry sample is input into the large model with the LoRA structure algorithm added for training to obtain a fine-tuned detection large model, including: Input the standardized jewelry sample into the large model with the LoRA structure algorithm added to obtain the prediction result; Calculate the loss function based on the prediction results and standardized sample annotations; Adjusting parameters of the LoRA structure algorithm according to the loss function; Continue to input the standardized jewelry sample into the large model with the LoRA structural algorithm added to obtain the prediction result, calculate the loss function according to the prediction result and the standardized sample annotation, and adjust the parameters of the LoRA structural algorithm according to the loss function until the loss function is minimized to obtain the fine-tuned detection large model.

6. The method according to claim 4, characterized in that The standardized jewelry samples are input into the large model with the LoRA structure algorithm added for training, and blind sample reasoning verification is performed to obtain a fine-tuned detection large model, including: Selecting a standardized jewelry sample formed by jewelry tested in history from the standardized jewelry sample as a first data set; Inputting the first data set into a large model with the LoRA structure algorithm added for training; From the remaining standardized jewelry samples, a first preset number of standardized jewelry samples are selected as blind samples for testing, and standardized jewelry samples with errors in the test are used as a second data set; Inputting the second data set into the large model with the LoRA structure algorithm added for training; From the remaining standardized main samples, a second preset number of standardized jewelry samples are selected as blind samples for testing, and the standardized jewelry samples that have tested errors are used as a third data set; Continue to iterate the training of the above steps until the prediction accuracy of the blind sample for style, cut and color is greater than a preset accuracy threshold, and then stop the training.

7. The method according to any one of claims 1 to 3, characterized in that: After obtaining the image of the jewelry to be tested, the method further includes: sending the image number of the jewelry to be tested and the image to be tested to a laboratory management system; After analyzing the test results, the style, cut and color of the jewelry to be tested are filled in the corresponding position in the laboratory management system, including: Determine the image number of the jewelry to be tested corresponding to the test result; Determine the corresponding image number in the laboratory management system according to the image number; The style, cut and color of the jewelry to be tested are filled into the corresponding position of the image number of the laboratory management system through the API interface.

8. A jewelry auxiliary detection device based on LoRA fine-tuning large model, characterized in that: The device comprises: An acquisition module is used to acquire an image of the jewelry to be detected; the image to be detected is obtained by photographing the jewelry to be detected placed in the darkroom environment using a camera device set in the darkroom environment; A detection module, used for inputting the image to be detected into the fine-tuned detection model to obtain a detection result; the detection result includes the style, cut and color corresponding to the jewelry to be detected; the fine-tuned detection model is obtained by fine-tuning the large model based on a standardized jewelry sample using a fine-tuning technology; the standardized jewelry sample includes a jewelry sample image and a standardized sample annotation; the standardized jewelry sample image includes a jewelry sample image obtained by photographing the sample jewelry using a camera device in a darkroom environment; the standardized sample annotation includes a description of the style, cut and color; The analysis and filling module is used to fill in the style, cut and color of the jewelry to be tested into the corresponding position in the laboratory management system after analyzing the test results.

9. A computer device, characterized in that: include: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform the operation of the jewelry assisted detection method based on the LoRA fine-tuning large model as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that: The storage medium stores at least one executable instruction. When the executable instruction is executed on a computer device, the computer device executes the operation of the jewelry assisted detection method based on the LoRA fine-tuning large model as described in any one of claims 1 to 7.

Images