Method, system and device for identifying product information and storage medium

Abstract

The invention discloses a method, system and device for identifying product information and a storage medium, and mainly relates to the technical field of natural language processing. Comprising the following steps: creating a project by using a ruoyi framework, and locally deploying a Mini CPM-V model by using an olan model tool; carrying out local training on a model fusion algorithm by using an Anything LLM knowledge base tool; and performing interface testing on the trained model. The method has the beneficial effects that the method can be accurately applied to the field of product identification while the output range is narrowed.

Description

Technical Field

[0001] The present invention relates to the field of natural language processing technology, and in particular to a method, system, device and storage medium for identifying product information. Background Art

[0002] AI recognition technology is an important branch in the field of artificial intelligence. It covers multiple aspects such as image recognition, speech recognition, and text recognition. It has been applied to autonomous driving, medical image analysis, handwriting recognition and other fields. With the continuous advancement of technology and the continuous expansion of application scenarios, AI recognition technology will play an important role in more fields and bring more convenience and efficiency to people's lives and work.

[0003] However, existing AI image recognition technologies, such as the SigLip-400M model, can recognize uploaded images and give relevant image descriptions. The drawback is that the language in it is only used to describe images and cannot be actually applied to specific projects, such as medical image recognition and image product recognition. The MiniCPM-V model combines the MiniCPM language model with the SigLip-400M image model. Natural language can be input at the same time as the image. The model will output the corresponding image description based on the natural language input by the user. The disadvantage is that its output range is large and cannot be accurately used in the field of product recognition.

[0004] Therefore, there is an urgent need for a method, system, device and storage medium for identifying product information to solve the above problems. Summary of the invention

[0005] The object of the present invention is to provide a method, system, device and storage medium for identifying product information, which can achieve narrowing the output range while being accurately applied to the field of product identification.

[0006] In order to achieve the above object, the present invention is implemented through the following technical solutions:

[0007] In one aspect, a method for identifying product information is provided, comprising the following steps:

[0008] S1: Use the ruoyi framework to create a project and use the ollama model tool to locally deploy the MiniCPM-V model;

[0009] S2: Locally train the model integration algorithm using the Anything LLM knowledge base tool;

[0010] S3: Perform interface testing on the trained model.

[0011] Preferably, the step S2 specifically includes the following steps:

[0012] S21: After deploying the MiniCPM-V model, input the target image;

[0013] S22: Determine whether the path of the target image is empty;

[0014] S23: If the path of the target image is judged to be empty, the process is terminated; if the path of the target image is judged to be not empty, the Anything model training is performed.

[0015] Preferably, the Anything LLM knowledge base tool integrates the model into the algorithm specifically as follows:

[0016] Use Spring AI to integrate ollama into the backend, rewrite the Prompt class when calling the interface, customize the input language format, convert the image into bytes and upload it to MiniCPM-V for recognition processing, specify the returned product name and product information through natural language, and expose the interface to the frontend.

[0017] Preferably, the Anything LLM knowledge base tool also includes:

[0018] According to the string behind the @GetMapping annotation of the method, define the location of the image, load the image and package it into an object, that is, the BufferedImage type object image, define the byte array output stream, and write the image object into the stream. At this time, baos has written the image content, and a byte array object imageBytes is returned through the toByteArray method of baos;

[0019] The array object imageBytes is used as the second parameter of the new Media constructor to generate the userMessage object and the response data. The first parameter of the call method is the prompt constructor New Prompt(List.of(userMessage). The second parameter is the chain call of OllamaOptions, that is, the create() method returns the value and then calls withModel(), whose parameter is minicpm-v. The final return value is the ChatResponse object response. The final method return value is the Generation object response.getResult().

[0020] Preferably, the step S3 specifically includes: testing the interface address through the postman tool.

[0021] Preferably, it also includes:

[0022] S4: After the test is completed, the output description is combined with natural language, and the front-end rendering is performed. If the output description meets the threshold, the process ends. If the output description does not meet the threshold, it returns to step S21 and re-inputs the target image

[0023] In another aspect, a system for identifying product information is provided, comprising:

[0024] Model creation module, used to: create a project using the ruoyi framework and deploy the MiniCPM-V model locally using the ollama model tool;

[0025] Model training module, used to: locally train the model integration algorithm using the Anything LLM knowledge base tool;

[0026] The interface testing module is used to perform interface testing on the trained model.

[0027] Preferably, the step S1 specifically comprises:

[0028] The voltage transformer is provided with a voltage sensor and a temperature sensor, and the current transformer is provided with a current sensor. Voltage information, temperature information and current information are collected respectively through the voltage sensor, the temperature sensor and the current sensor.

[0029] On the other hand, a device for identifying product information is provided, comprising a processor and a memory for storing a computer program, wherein when the processor executes the computer program, the steps of any one of the above methods are implemented.

[0030] On the other hand, a computer-readable storage medium is provided, characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is executed by one or more processors, the steps of any of the methods described above are implemented.

[0031] Compared with the prior art, the beneficial effects of the present invention are:

[0032] By retraining the existing AI model and transforming the large model, it is applicable to all image product information upload projects. You only need to call the interface to upload the image to independently identify the image and complete the conversion of image information into product information upload. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 It is a flow chart of the method of the present invention.

[0034] Figure 2 It is a schematic diagram of the system structure of the present invention.

[0035] Figure 3 It is a schematic diagram of the output result of the present invention. DETAILED DESCRIPTION

[0036] The present invention will be further described below in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not intended to limit the scope of the present invention. In addition, it should be understood that after reading the content taught by the present invention, those skilled in the art can make various changes or modifications to the present invention, and these equivalent forms fall within the scope limited by the application equally.

[0037] In the present invention, terms such as "upper", "lower", "left", "right", "front", "back", "vertical", "horizontal", "side", "bottom", etc. indicate directions or positional relationships based on the directions or positional relationships shown in the accompanying drawings. They are relational words determined only for the convenience of describing the structural relationships of the various parts or elements of the present invention, and do not specifically refer to any part or element in the present invention and should not be understood as limitations on the present invention.

[0038] In the present invention, terms such as "fixed connection", "connected", "connection", etc. should be understood in a broad sense, indicating that it can be fixedly connected, integrally connected or detachably connected; it can be directly connected or indirectly connected through an intermediate medium. Relevant scientific research or technical personnel in this field can determine the specific meanings of the above terms in the present invention according to specific circumstances, and they should not be understood as limiting the present invention.

[0039] Example:

[0040] like Figure 1 , this embodiment provides a method for identifying product information, comprising the following steps:

[0041] S1: Use the ruoyi framework to create a project and use the ollama model tool to locally deploy the MiniCPM-V model;

[0042] S2: Locally train the model integration algorithm using the Anything LLM knowledge base tool;

[0043] S3: Perform interface testing on the trained model.

[0044] The step S2 specifically includes the following steps:

[0045] S21: After deploying the MiniCPM-V model, input the target image;

[0046] S22: Determine whether the path of the target image is empty;

[0047] S23: If the path of the target image is judged to be empty, the process is terminated; if the path of the target image is judged to be not empty, the Anything model training is performed.

[0048] The Anything LLM knowledge base tool integrates the model into the algorithm specifically as follows:

[0049] Use Spring AI to integrate ollama into the backend, rewrite the Prompt class when calling the interface, customize the input language format, convert the image into bytes and upload it to MiniCPM-V for recognition processing, specify the returned product name and product information through natural language, and expose the interface to the frontend. The Anything LLM knowledge base tool also includes the following in the model integration algorithm:

[0050] @GetMapping(" / img")

[0051] public Generation getImg(String img)throws IOException{

[0052] String imagePath="D:\\zszy\\file\\"+img;

[0053] BufferedImageimage=ImageIO.read(new File(imagePath));

[0054] ByteArrayOutputStreambaos=newByteArrayOutputStream();

[0055] ImageIO.write(image,"jpg",baos);

[0056] byte[]imageBytes=baos.toByteArray();

[0057] var userMessage=new UserMessage("Generate a product name and product information based on the item in the picture",

[0058] List.of(newMedia(MimeTypeUtils.IMAGE_PNG,imageBytes)));

[0059] ChatResponse response=ollamaChatModel.call(NewPrompt(List.of(userMessage),OllamaOptions.create().withModel("minicpm-v:latest")));

[0060] return response.getResult();

[0061] The following code is explained line by line:

[0062] According to the string behind the @GetMapping annotation of the method, define the location of the image to prepare for the use of imagePath later.

[0063] BufferedImage image=ImageIO.read(new File(imagePath)); This line of code loads the image and packages it into an object, that is, the BufferedImage type object image, ByteArrayOutputStream baos=new ByteArrayOutputStream(); This line of code defines the byte array output stream, which is prepared for the subsequent image object to be written into the stream, that is, ImageIO.write(image,"jpg",baos); This line of code implements writing the image into the stream, baos has already written the image content at this time, byte[]imageBytes=baos.toByteArray(); This line of code returns a byte array object imageBytes through the toByteArray method of baos.

[0064] The array object imageBytes is used as the second parameter of the new Media constructor to generate the userMessage object, ChatResponse response = ollamaChatModel.call(NewPrompt(List.of(userMessage), OllamaOptions.create().withModel("minicpm-v:latest"))); This line of code is the core of the entire method, and its purpose is to generate response data. The first parameter of the call method is the prompt constructor NewPrompt(List.of(userMessage), and the second parameter is the chain call of OllamaOptions, that is, the create() method returns the value and then calls withModel(), whose parameter is minicpm-v, and the following :latest is the latest version number. The final return value is the ChatResponse object response, and the final method return value is the Generation object response.getResult().

[0065] Use the postman interface tool to test the above code as follows:

[0066]

[0067] The above is tested by the postman tool. The interface address is: http: / / localhost:9090 / img. The following img=xxx.jpg is a parameter of String type. After submitting the request interface, the return value is a string in JSON format.

[0068] like Figure 2 As shown, this embodiment also provides a system for identifying product information, including:

[0069] Model creation module, used to: create a project using the ruoyi framework and deploy the MiniCPM-V model locally using the ollama model tool;

[0070] Model training module, used to: locally train the model integration algorithm using the Anything LLM knowledge base tool;

[0071] The interface testing module is used to perform interface testing on the trained model.

Claims

1. A method for identifying product information, characterized in that: The following steps are involved: S1: Use the ruoyi framework to create a project and use the ollama model tool to locally deploy the MiniCPM-V model; S2: Locally train the model integration algorithm using the Anything LLM knowledge base tool; S3: Perform interface testing on the trained model.

2. A method for identifying product information according to claim 1, characterized in that: The step S2 specifically includes the following steps: S21: After deploying the MiniCPM-V model, input the target image; S22: Determine whether the path of the target image is empty; S23: If the path of the target image is judged to be empty, the process is terminated; if the path of the target image is judged to be not empty, the Anything model training is performed.

3. A method for identifying product information according to claim 2, characterized in that: The Anything LLM knowledge base tool integrates the model into the algorithm specifically as follows: Use Spring AI to integrate ollama into the backend, rewrite the Prompt class when calling the interface, customize the input language format, convert the image into bytes and upload it to MiniCPM-V for recognition processing, specify the returned product name and product information through natural language, and expose the interface to the frontend.

4. A method for identifying product information according to claim 3, characterized in that: The Anything LLM knowledge base tool also includes the following model integration algorithms: According to the string behind the @GetMapping annotation of the method, define the location of the image, load the image and package it into an object, that is, the BufferedImage type object image, define the byte array output stream, and write the image object into the stream. At this time, baos has written the image content, and a byte array object imageBytes is returned through the toByteArray method of baos; The array object imageBytes is used as the second parameter of the new Media constructor to generate the userMessage object and the response data. The first parameter of the call method is the prompt constructor New Prompt(List.of(userMessage). The second parameter is the chain call of OllamaOptions, that is, the create() method returns the value and then calls withModel(), whose parameter is minicpm-v. The final return value is the ChatResponse object response. The final method return value is the Generation object response.getResult().

5. A method for identifying product information according to claim 1, characterized in that: The step S3 specifically includes: testing the interface address through the postman tool.

6. A method for identifying product information according to claim 2, characterized in that: Also includes: S4: After the test is completed, the output description is combined with natural language, and the front-end rendering is performed. If the output description meets the threshold, the process is terminated. If the output description does not meet the threshold, it returns to step S21 and re-inputs the target image.

7. A system for identifying product information, characterized in that: include: Model creation module, used to: create a project using the ruoyi framework and deploy the MiniCPM-V model locally using the ollama model tool; Model training module, used to: locally train the model integration algorithm using the Anything LLM knowledge base tool; The interface testing module is used to perform interface testing on the trained model.

8. A device for identifying product information, characterized in that: The method comprises a processor and a memory for storing a computer program, wherein when the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by one or more processors, the steps of the method according to any one of claims 1 to 6 are implemented.

Images