Work identification processing method and device

Abstract

The embodiment of the invention provides a work recognition processing method and device, and the method comprises the steps: generating a recognition prompt word for a work image, uploaded by a user terminal, of a real work in the recognition processing process of the real work, the method comprises the steps of obtaining a work image, inputting an identification prompt word and the work image into a large language model for work identification and object association retrieval, obtaining a physical object represented by a real work, returning the physical object to a user terminal, receiving a selected target physical object, generating knowledge data of the target physical object, and returning the knowledge data to the user terminal. Therefore, linkage between the physical works and the physical objects and corresponding knowledge learning are realized.

Description

Technical Field

[0001] This document relates to the field of data processing technologies, and in particular, to a method and apparatus for work recognition and processing. Background Art

[0002] With the continuous development and popularization of artificial intelligence technologies and hardware devices, the scope of application of various services provided based on artificial intelligence technologies and hardware devices is also becoming wider and wider. In this context, various ways of application interaction through artificial intelligence technologies and hardware devices have emerged, such as application programs for education through AI (Artificial Intelligence), and services for question dialogue and knowledge retrieval through AI. However, as the number of application programs accessing AI services increases, the competition among application providers is also intensifying. In this context, how to increase the application frequency of AI services in actual life scenarios has become the focus of attention of all parties. Summary of the Invention

[0003] One or more embodiments of this specification provide a method for work recognition and processing, including: obtaining a work image of a physical work uploaded by a user terminal, and generating an identification prompt word for work recognition and object association retrieval of the work image. Inputting the identification prompt word and the work image into a large language model for work recognition and object association retrieval to obtain the physical object represented by the physical work. Returning the physical object to the user terminal, and receiving the target physical object selected by the user terminal from the physical objects. Generating knowledge data of the target physical object and returning it to the user terminal.

[0004] One or more embodiments of this specification provide another method for work recognition and processing, including: collecting a work image of a physical work and uploading it to a server to call a large language model for work recognition and object association retrieval based on the work image and an identification prompt word to obtain a physical object. Receiving and displaying the physical object returned by the server, and submitting the target physical object selected from the physical objects to the server. Performing interactive display of the knowledge data according to the knowledge data of the target physical object returned by the server.

[0005] One or more embodiments of this specification provide a work recognition processing device, including: a prompt word generation module configured to obtain a work image of a physical work uploaded by a user terminal and generate an identification prompt word for performing work recognition and object association retrieval on the work image. An identification retrieval module configured to input the identification prompt word and the work image into a large language model for work recognition and object association retrieval to obtain a physical object represented by the physical work. A physical object return module configured to return the physical object to the user terminal and receive a target physical object selected by the user terminal from the physical object. A knowledge generation module configured to generate knowledge data of the target physical object and return it to the user terminal.

[0006] One or more embodiments of this specification provide another work recognition processing device, including: a work image acquisition module configured to acquire a work image of a physical work and upload it to a server to call a large language model according to the work image and an identification prompt word for work recognition and object association retrieval to obtain a physical object. A physical object selection module configured to receive and display the physical object returned by the server and submit the selected target physical object in the physical object to the server. A knowledge data display module configured to perform interactive display of the knowledge data according to the knowledge data of the target physical object returned by the server.

[0007] One or more embodiments of this specification provide a work recognition processing device, including: a processor; and a memory configured to store computer-executable instructions, where the computer-executable instructions, when executed, cause the processor to: obtain a work image of a physical work uploaded by a user terminal and generate an identification prompt word for performing work recognition and object association retrieval on the work image. Input the identification prompt word and the work image into a large language model for work recognition and object association retrieval to obtain a physical object represented by the physical work. Return the physical object to the user terminal and receive a target physical object selected by the user terminal from the physical object. Generate knowledge data of the target physical object and return it to the user terminal.

[0008] One or more embodiments of this specification provide another work recognition processing device, including: a processor; and a memory configured to store computer-executable instructions, which, when executed, cause the processor to: collect a work image of a physical work and upload it to a server, and call a large language model according to the work image and recognition prompt words to perform work recognition and object association retrieval to obtain a physical object. Receive and display the physical object returned by the server, and submit the selected target physical object in the physical object to the server. According to the knowledge data of the target physical object returned by the server, perform interactive display of the knowledge data.

[0009] One or more embodiments of this specification provide a computer-readable storage medium for storing computer-executable instructions, which, when executed, implement the following process: obtain a work image of a physical work uploaded by a user terminal, and generate recognition prompt words for performing work recognition and object association retrieval on the work image. Input the recognition prompt words and the work image into a large language model to perform work recognition and object association retrieval to obtain the physical object represented by the physical work. Return the physical object to the user terminal, and receive the target physical object selected by the user terminal from the physical objects. Generate knowledge data of the target physical object and return it to the user terminal.

[0010] One or more embodiments of this specification provide another computer-readable storage medium for storing computer-executable instructions, which, when executed, implement the following process: collect a work image of a physical work and upload it to a server, and call a large language model according to the work image and recognition prompt words to perform work recognition and object association retrieval to obtain a physical object. Receive and display the physical object returned by the server, and submit the selected target physical object in the physical object to the server. According to the knowledge data of the target physical object returned by the server, perform interactive display of the knowledge data. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate the technical solutions in one or more embodiments of this specification or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments recorded in this specification. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings; Figure 1 It is a schematic diagram of the implementation environment of a work recognition processing method provided by one or more embodiments of this specification; Figure 2A processing flow chart of a work recognition processing method provided for one or more embodiments of this specification; Figure 3 A schematic diagram of a physical work collection page provided for one or more embodiments of this specification; Figure 4 A schematic diagram of a physical object selection page provided for one or more embodiments of this specification; Figure 5 A schematic diagram of a knowledge text page provided for one or more embodiments of this specification; Figure 6 A schematic diagram of a work archive page provided for one or more embodiments of this specification; Figure 7 A processing flow chart of a work recognition processing method applied to a work recognition service scenario provided for one or more embodiments of this specification; Figure 8 Another processing flow chart of a work recognition processing method provided for one or more embodiments of this specification; Figure 9 A schematic diagram of an embodiment of a work recognition processing device provided for one or more embodiments of this specification; Figure 10 Another schematic diagram of an embodiment of a work recognition processing device provided for one or more embodiments of this specification; Figure 11 A schematic diagram of the structure of a work recognition processing device provided for one or more embodiments of this specification Figure 12 Another schematic diagram of the structure of a work recognition processing device provided for one or more embodiments of this specification. Detailed implementation manners

[0012] In order to enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the following will clearly and completely describe the technical solutions in one or more embodiments of this specification with reference to the accompanying drawings in one or more embodiments of this specification. Obviously, the described embodiments are only a part of the embodiments of this specification, rather than all the embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of this document.

[0013] The work recognition processing method provided by one or more embodiments of this specification is applicable to an implementation environment of a work recognition processing system. Referring to Figure 1 , this implementation environment at least includes: A user terminal 101 and a server 102; Among them, the user terminal 101 is used to collect the work images of physical works and upload them to the server 102, and cooperate with the server 102 for interaction and learning of knowledge data; the user terminal 101 can specifically be a mobile phone, a personal computer, a tablet computer, an e-book reader, a device for information interaction based on VR (Virtual Reality), AR (Augmented Reality), a vehicle-mounted terminal, an IoT device, a wearable intelligent device, a laptop computer, a desktop computer, and so on; The server 102 is used to perform work recognition and object association retrieval on the work images uploaded by the user terminal 101, and cooperate with the user terminal 101 for interaction and learning of knowledge data. The server 102 can be a single server, or a server cluster composed of several servers, or one or more cloud servers in a cloud computing platform.

[0014] In this implementation environment, during the process of the user terminal 101 and the server 102 cooperating to perform work recognition processing, for the work images of physical works collected and uploaded by the user terminal 101, based on generating recognition prompt words for work recognition and object association retrieval of the work images, the server 102 inputs the recognition prompt words and the work images into a large language model to perform work recognition and object association retrieval, obtains the physical objects represented by the physical works, selects the target physical objects in cooperation with the user terminal 101, and finally generates the knowledge data of the target physical objects and returns them to the user terminal, so as to realize the knowledge learning interaction of the physical objects represented by the physical works on the basis of recognizing the physical works through the work images.

[0015] One or more embodiments of a work recognition processing method provided in this specification are as follows: Refer to Figure 2 In this embodiment, the work recognition processing method provided, the method specifically includes steps S202 to S208.

[0016] Step S202, obtain the work images of the physical works uploaded by the user terminal, and generate recognition prompt words for work recognition and object association retrieval of the work images.

[0017] The physical work in this embodiment refers to a work with an entity form obtained by using substances in the physical world. Specifically, it can be an entity form work obtained by using one or more items, one or more tools, and / or one or more materials in the physical world, such as an entity form model / physical model. Optionally, the physical work includes: a physical model obtained by performing physical construction according to items, tools, and / or materials. The physical construction methods can include physical model building, physical model combination, and / or physical model construction.

[0018] Specifically, in the process of constructing a physical work using items, tools, and / or materials, the construction method can be to build a physical work by using items, tools, and / or materials, or to build a physical work by using a part of the items, tools, and / or materials to build another part of the items, tools, and / or materials. In this case, the physical work can also be called a physical construction work or a construction work. For example, a physical model can be obtained by building a physical model using items, tools, and / or materials; Or, the construction method can also be to combine items, tools, and / or materials to obtain a physical work, or to use a part of the items, tools, and / or materials to combine another part of the items, tools, and / or materials to obtain a physical work. In this case, the physical work can also be called a combined physical work or a combination work. For example, a physical model can be obtained by combining a physical model using items, tools, and / or materials; Or, the construction method can also be to build a physical work by using items, tools, and / or materials, or to use a part of the items, tools, and / or materials to build another part of the items, tools, and / or materials to obtain a physical work. In this case, the physical work can also be called a constructed physical work or a construction work. For example, a physical model can be obtained by using For example, a building block model obtained by building with building block toys. Different forms of the building block model can be used to represent different forms of buildings, vehicles, animals, plants, mountains, rivers, seas, or other living or non-living things in the physical world; or, A physical model obtained by building with magnetic toys. Different forms of the physical model can also be used to represent different forms of buildings, vehicles, animals, plants, mountains, rivers, seas, or other living or non-living things in the physical world; or, A physical model obtained by combining materials such as clay or plasticine. Different forms of the physical model can also be used to represent different forms of buildings, vehicles, animals, plants, mountains, rivers, seas, or other living or non-living things in the physical world; or, A physical work or a physical model obtained by building with paper materials / wood materials / paper materials and glue / wood materials and glue / wood materials, glue, and tools. Different forms of the physical work or the physical model can also be used to represent different forms of buildings, vehicles, animals, plants, mountains, rivers, seas, or other living or non-living things in the physical world.

[0019] In an actual scenario, the user who constructs a physical work can be a teenage user. In this case, after the teenage user completes the construction of the physical work, the guardian user of the teenage user can collect an image of the physical work through a user terminal to obtain a work image. Or, if the teenage user owns a user terminal himself / herself, the teenage user can also collect an image of the physical work through the user terminal to obtain a work image. In addition, the user who constructs a physical work can be an adult user. After the adult user completes the construction of the physical work, the adult user can collect an image of the physical work through the user terminal to obtain a work image.

[0020] Specifically in implementation, an image of the physical work is collected through a user terminal. After the image collection is completed and the work image is obtained, the user terminal uploads the work image. Correspondingly, the work image of the physical work uploaded by the user terminal is obtained here. Specifically, to collect an image of the physical work to obtain a work image, the image collection can be performed by accessing an application installed on the user terminal. After the image collection is completed, the user terminal can upload the work image. Or, the image collection can also be performed by accessing a service or subroutine provided by the application. After the image collection is completed, the user terminal can upload the work image.

[0021] Furthermore, after obtaining the work image of the physical work uploaded by the user terminal, it is necessary to identify the work image and perform relevant knowledge learning based on the work identification. The process of identifying the work image can be achieved by calling a large language model. Specifically, before calling the large language model, an identification prompt word for work identification and object association retrieval of the work image is generated. For example, the generated identification prompt word contains a description text for work identification and also contains a description text for object association retrieval. These two parts together form the identification prompt word (identification prompt text) for work identification and object association retrieval.

[0022] Step S204, input the identification prompt word and the work image into the large language model for work identification and object association retrieval to obtain the physical object represented by the physical work.

[0023] In this embodiment, there is a certain relationship between the physical work and the physical object that actually exists in the physical world. This relationship can be that the physical work constructed by the user can represent the physical object that actually exists in the physical world, or the physical work constructed by the user is used to refer to the physical object that actually exists in the physical world. For example, if the color and structure of the building block model built by the user are similar to those of a certain building in the physical world, then this building block model can represent or refer to the building in the physical world. Another example is that if the wooden car model built by the user is similar to an ambulance in the physical world, then this building block model can represent or refer to the ambulance in the physical world.

[0024] Work identification and object association retrieval are performed here. Its function is to identify and recognize physical works, and based on the information obtained from the recognition, determine which real physical object or objects in the physical world the physical work represents or refers to. That is, the purpose of work identification and object association retrieval is to determine the physical object represented or referred to by the physical work.

[0025] In specific implementation, during the process of inputting the recognition prompt and the work image into the large language model for work identification and object association retrieval, in the work identification link, the work image can be recognized to obtain recognition information. In the object association retrieval link, based on the obtained recognition information, retrieve the physical object that has a representational relationship with the physical work as the physical object represented by the physical work. The large language model refers to a work recognition model that can perform corresponding recognition processing on the input work image based on the input recognition prompt, and this large language model is a multi-modal model that supports text and image input. The large language model can adopt a natural language model with a neural network architecture containing a large number of parameters, and can also adopt a pre-trained large language model (Large Language Model, LLM) or an open-source large language model. Or, it can also be obtained by fine-tuning the base large language model or the large language model.

[0026] Specifically, during the work identification process, in order to improve the recognition success rate and accuracy of recognizing the work image, work image blocks can be extracted from the work image. The extracted work image blocks are the image blocks corresponding to the physical work in the work image. It is also possible to perform recognition on the work image from multiple dimensions based on the work image blocks to obtain corresponding recognition information. In an optional implementation manner provided in this embodiment, work identification includes: Perform shape element recognition on the work image blocks extracted from the work image to obtain shape elements, and perform color relationship analysis on the work image blocks and / or shape elements to obtain color relationships; Determine the associated physical objects of the historical target physical object based on the historical target physical object represented by the historical physical work recorded in the user recognition record.

[0027] In the specific execution process, the work image blocks of the physical work can be extracted from the work image through the image segmentation module or image segmentation algorithm configured by the large language model. The shape elements can also be obtained by identifying the shape elements of the work image blocks through the shape recognition module configured by the large language model. The color relationship can also be obtained by analyzing the color relationship of the work image blocks and / or shape elements through the color relationship module configured by the large language model. In addition, the associated physical object of the historical target physical object represented by the historical physical work recorded in the user recognition record can be determined through the historical retrieval module configured by the large language model. The obtained shape elements, color relationships, and associated physical objects are used as recognition information. Among them, the user recognition record refers to the record of the user's terminal or the user's previous recognition of physical works. Considering that the user recognition record belongs to the user's privacy to a certain extent, the user's authorization can be obtained before obtaining the user recognition record.

[0028] It should be noted that the above-mentioned work recognition is carried out in three dimensions of shape elements, color relationships, and historical recognition objects for the work image blocks. In the actual scenario, the work image blocks can also be recognized in any one or any two of the three dimensions of shape elements, color relationships, and historical recognition objects according to actual needs to obtain corresponding recognition information. For example, the work recognition includes: identifying the shape elements by identifying the shape elements of the work image blocks extracted from the work image, and analyzing the color relationship of the work image blocks and / or shape elements to obtain the color relationship. The shape elements and color relationships can be used as recognition information. Another example is that the work recognition includes identifying the shape elements by identifying the work image blocks extracted from the work image, or the work recognition includes analyzing the color relationship of the work image blocks and / or shape elements to obtain the color relationship.

[0029] Based on this, in the object association retrieval process, based on the recognition information obtained from the work recognition, the retrieval is carried out. Specifically, it is to retrieve the physical objects that have a representational relationship with the physical work. Specifically, the physical object retrieval can be carried out according to the shape elements, color relationships, and / or associated objects to obtain the physical object represented by the physical work.

[0030] In an optional implementation manner provided in this embodiment, the object association retrieval includes: Retrieving candidate physical objects that have a representational relationship with the physical work according to the shape elements, color relationships, and / or associated physical objects, and scoring each retrieved candidate physical object; Calculating the retrieval scores of each candidate physical object according to the obtained scores and score weights, and selecting at least one candidate physical object as the physical object according to the retrieval scores.

[0031] In the specific implementation process, candidate physical objects having a representational relationship with the physical work are retrieved according to shape elements, color relationships, and / or associated physical objects. For the retrieved candidate physical objects, the candidate physical objects can be scored from three dimensions: shape elements, color relationships, and / or associated physical objects. The scoring can specifically be a probability, which refers to the possibility of evaluating the candidate physical object as a physical object representing the physical work in the three dimensions of shape elements, color relationships, and / or associated physical objects. Alternatively, the scoring can also be obtained by using a scoring algorithm; Based on the scores of each candidate physical object in these three dimensions and combined with the respective scoring weights of these three dimensions, the weighted score of each candidate physical object can be obtained through weighted calculation as the retrieval score. After obtaining the retrieval score, the candidate physical objects can be sorted in descending order, and the top N candidate physical objects in the sorting can be selected as the physical objects representing the physical work.

[0032] In practical applications, in order to improve the accuracy of recognizing and processing the work image, three-dimensional modeling can also be performed on the work image blocks included in the work image, and based on this three-dimensional modeling, work recognition and object association retrieval can be performed. Specifically, in an optional implementation manner provided in this embodiment, work recognition and object association retrieval are implemented in the following manner: Three-dimensional modeling is performed on the work image blocks included in the work image to obtain a virtual work, and physical recognition is performed on the virtual work to obtain first recognition information; Shape element recognition, color relationship analysis, and / or associated object query are performed on the virtual work to obtain second recognition information; According to the first recognition information, the second recognition information, and the virtual work, physical objects having a representational relationship with the physical work are retrieved.

[0033] Among them, in the process of performing three-dimensional modeling on the work image blocks to obtain a virtual work, the three-dimensional modeling module configured by the large language model can be used to perform three-dimensional modeling on the work image blocks to obtain a virtual work, and the physical recognition module configured by the large language model can be used to perform physical recognition on the virtual work to obtain first recognition information.

[0034] Step S206, return the physical object to the user terminal, and receive the target physical object selected by the user terminal from the physical objects.

[0035] After obtaining the physical object representing the physical work through the large language model for work recognition and object association retrieval, the physical object is returned to the user terminal. Specifically, in the process of returning the physical object, based on obtaining the object identifier and object image of the physical object, the object identifier and object image of the physical object are returned to the user terminal. After the physical object is returned to the user terminal, the user terminal can make a selection among the returned physical objects. Specifically, the physical object that truly represents the current physical work is selected among the physical objects, and the physical object that truly represents the physical work selected is called the target physical object. After that, the user terminal submits the target physical object selected among the physical objects. Correspondingly, here the target physical object selected and submitted by the user terminal among the physical objects is received.

[0036] For example, after a teenage user builds a block model in the shape of a spire with building blocks, the parent user collects the model image of the block model through a subroutine of the application installed on the user terminal, as Figure 3 shown; after the model image of the block model is collected, the user terminal uploads the model image to the server in the background of the subroutine. After receiving the model image, the server performs work recognition and object association retrieval on the model image, and obtains 3 physical objects: the Eiffel Tower, the Tokyo Tower, and the TV Tower, and returns the names and images of these 3 physical objects to the user terminal. The user terminal receives and displays the names and images of these 3 physical objects returned by the server, as Figure 4 shown; For the names and images of the 3 displayed physical objects, the teenage user or the parent user can select the physical object that truly represents the block model built with building blocks among these 3 physical objects. If the teenage user builds the block model based on the Eiffel Tower as the prototype or wants to build a block model of the Eiffel Tower, then the physical object of the Eiffel Tower among the 3 physical objects can be selected and the selected physical object is submitted to the server.

[0037] Step S208: Generate knowledge data of the target physical object and return it to the user terminal.

[0038] Specifically in implementation, after obtaining the target physical object submitted by the user terminal, that is, after obtaining the target physical object that truly represents the physical work submitted by the user terminal, generate knowledge data of the target physical object and return it to the user terminal. In this way, by returning the knowledge data of the target physical object to the user terminal, the user who constructs the physical work can learn the knowledge data of the target physical object and also learn relevant knowledge during the construction process of the physical work.

[0039] In an alternative implementation provided in this embodiment, generating knowledge data of a target physical object includes: generating a learning knowledge text of the target physical object and performing voice conversion on the learning knowledge text to obtain learning knowledge voice.

[0040] Specifically, in the process of generating the learning knowledge text of the target physical object, the learning knowledge text can be generated by calling the corresponding knowledge generation interface, or the learning knowledge text can also be obtained by inputting the target physical object into a knowledge generation model. Optionally, the learning knowledge text includes: learning knowledge data or teaching knowledge data provided to adolescent users.

[0041] In the specific execution process, after the knowledge data of the generated target physical object is returned to the user terminal, after the user terminal receives the returned knowledge data of the target physical object, it performs interactive display of the knowledge data. Taking the above building block model as an example, the learning knowledge text of the building block model displayed on the user terminal is as Figure 5 shown, and the specific learning knowledge text displayed is: "The Eiffel Tower is a landmark building and a popular tourist attraction in Paris, France. It was completed in 1889 to celebrate the 100th anniversary of the French Revolution. The height of this tower is 330 meters and it was once the tallest building in the world. It is composed of 18,038 iron parts and weighs about 10,100 tons. Its unique structure has become a masterpiece of engineering art." It should be noted that in addition to the above-provided method of generating the learning knowledge text of the target physical object, teaching knowledge data or other types of knowledge data of the target physical object can also be generated, such as knowledge data like songs or ancient poems. That is, generating the learning knowledge text of the target physical object and performing voice conversion on the learning knowledge text to obtain learning knowledge voice can be replaced by: generating the teaching knowledge data of the target physical object and performing voice conversion on the teaching knowledge data to obtain teaching knowledge voice.

[0042] In practical applications, after generating the knowledge data of the target physical object and returning it to the user terminal, the uploaded work image can also be archived. Specifically, in an alternative implementation provided in this embodiment, if an archive instruction for the work image is detected, a work archive image is generated and stored based on the work image and the object image of the target physical object. Specifically, during the storage process, the work archive image can be stored in the user storage space, and this user storage space can be the user storage space provided by an application, a service provided by the application, or a subroutine.

[0043] On this basis, after generating the archived image of the physical Zouping's work and storing it in the user storage space, the user can access the archived image of the work stored in the user storage space through the user terminal. Still taking the above building block model as an example, during the process of the user storing the archived image of the building block model in the user storage space, the archived image of the building block model displayed on the user terminal is as Figure 6 shown.

[0044] In addition, to enhance the interaction between users based on the recognition of physical works and knowledge learning interaction, the user can also share the work image or the archived image of the work. The user who receives the share can also comment on the shared work image or the archived image of the work. Specifically, after generating the knowledge data of the target physical object and returning it to the user terminal, in an optional implementation provided in this embodiment, if a sharing instruction for the work image or the archived image of the work is detected, a work sharing message is generated and sent according to the selected sharing channel. According to the comment information submitted after the work sharing message is triggered, the comment information is associated with the work image or the archived image of the work.

[0045] In this embodiment, in order to improve the enthusiasm of users to participate in the recognition of physical works and knowledge learning interaction, in an optional implementation provided in this embodiment, if it is detected that the archived images of works in the same work category in the user storage space meet the task trigger condition, interaction data is generated according to the interaction materials and sent to the user terminal. Optionally, the task trigger condition includes: the position relationship of the geographical areas of the buildings corresponding to the building model works under the building model category meets the preset position relationship.

[0046] In summary, the work recognition processing method provided in this embodiment, during the process of work recognition and knowledge learning interaction for physical works, obtains the work image of the physical work uploaded by the user terminal, and generates an identification prompt word for work recognition and object association retrieval of the work image. By inputting the identification prompt word and the work image into the large language model for work recognition and object association retrieval, the physical object represented by the physical work is obtained, and by returning the physical object to the user terminal and receiving the target physical object selected by the user terminal from the physical objects, the target physical object truly represented by the physical work is determined. On this basis, the knowledge data of the target physical object is generated and returned to the user terminal. By returning the knowledge data of the target physical object to the user terminal, the user who constructs the physical work can learn the knowledge data of the target physical object and also learn relevant knowledge during the construction process of the physical work.

[0047] The above steps S202 to S208 provided in this embodiment can be executed by a server, which can specifically be a server for an application program, a server for a service, or a server for a subroutine; it should be noted that the above steps S202 to S208 executed by the server and the steps S802 to S806 executed by the user terminal in the following embodiment can cooperate with each other during the execution process. Therefore, when reading this embodiment, please refer to the corresponding content of steps S802 to S806 provided in the following method embodiment, and when reading the following method embodiment, please refer to the corresponding content of steps S202 to S208 provided in this embodiment.

[0048] The following takes the application of a work recognition processing method provided in this embodiment in a work recognition service scenario as an example, and combines Figure 7 , to further illustrate the work recognition processing method provided in this embodiment. See Figure 7 , the work recognition processing method applied to the work recognition service scenario specifically includes the following steps.

[0049] Step S704, receive the work image of the physical work uploaded by the user terminal.

[0050] Step S706, generate an identification prompt word for performing work recognition and object association retrieval on the work image.

[0051] Step S708, input the identification prompt word and the work image into a large language model for work recognition and object association retrieval to obtain multiple physical objects represented by the physical work.

[0052] Step S710, return multiple physical objects represented by the physical work to the user terminal.

[0053] Step S716, receive the target physical object selected by the user terminal from the physical objects.

[0054] Step S718, generate a learning knowledge text for the target physical object and return it to the user terminal.

[0055] It should be noted that any one step or any combination of steps among steps S704, steps S706 to S710, and steps S716 to S718 can be combined with any one step or any combination of steps among the above steps S202 to S208 according to the needs of implementation and deployment to form a new implementation method; in addition, according to the actual deployment needs, any one or any combination of technical features in steps S704, steps S706 to S710, and steps S716 to S718 can be selected and combined with any one or more technical features provided by the above steps S202 to S208 to form a new implementation method; or, any one or any combination of technical features in steps S704, steps S706 to S710, and steps S716 to S718 can also be replaced by any one or more technical feature combinations provided by the above steps S202 to S208 according to the actual deployment needs to form a new implementation method, which will not be elaborated here one by one.

[0056] In addition, it should also be noted that the above steps S704, steps S706 to S710, and steps S716 to S718 provided in this embodiment can be executed by the server. It should be noted that the above steps S704, steps S706 to S710, and steps S716 to S718 executed by the server can cooperate with the steps S702, steps S712 to S714, and steps S720 to S722 executed by the user terminal in the following embodiment during the execution process. Therefore, when reading this embodiment, please refer to the corresponding content of steps S702, steps S712 to S714, and steps S720 to S722 provided in the following method embodiment, and when reading the following method embodiment, please refer to the corresponding content of steps S704, steps S706 to S710, and steps S716 to S718 provided in this embodiment.

[0057] Another embodiment of the work recognition processing method provided in this specification is as follows: Refer to Figure 8 , the work recognition processing method provided in this embodiment can be applied to a user terminal, and the method specifically includes steps S802 to S806.

[0058] Step S802, collect the work image of the physical work and upload it to the server to call the large language model for work recognition and object association retrieval according to the work image and the recognition prompt word to obtain the physical object represented by the physical work.

[0059] The physical work described in this embodiment refers to a work with an entity form obtained by using substances in the physical world. Specifically, it can be an entity form work obtained by using one or more items, one or more tools, and / or one or more materials in the physical world, such as an entity form model / physical model. Optionally, the physical work includes: a physical model obtained by physical construction according to items, tools, and / or materials. The physical construction methods may include physical model building, physical model combination, and / or physical model construction.

[0060] Specifically, in the process of constructing a physical work using items, tools, and / or materials, the construction method can be to use the items, tools, and / or materials to build a physical work, or to use a part of the items, tools, and / or materials to build another part of the items, tools, and / or materials to obtain a physical work. In this case, the physical work can also be called a physical construction work or a construction work. For example, a physical model can be obtained by using items, tools, and / or materials to build a physical model; Or, the construction method can also be to combine the items, tools, and / or materials to obtain a physical work, or to use a part of the items, tools, and / or materials to combine another part of the items, tools, and / or materials to obtain a physical work. In this case, the physical work can also be called a combined physical work or a combination work. For example, a physical model can be obtained by using items, tools, and / or materials to combine a physical model; Or, the construction method can also be to use the items, tools, and / or materials to construct a physical work, or to use a part of the items, tools, and / or materials to construct another part of the items, tools, and / or materials to obtain a physical work. In this case, the physical work can also be called a constructed physical work or a construction work. For example, a physical model can be obtained by For example, a building block model obtained by building with building block toys. Different forms of the building block model can be used to represent different forms of buildings, vehicles, animals, plants, mountains, rivers, seas, or other living or non-living things in the physical world; or, A physical model obtained by building with magnetic toys. Different forms of the physical model can also be used to represent different forms of buildings, vehicles, animals, plants, mountains, rivers, seas, or other living or non-living things in the physical world; or, A physical model obtained by combining materials such as clay or plasticine. Different forms of the physical model can also be used to represent different forms of buildings, vehicles, animals, plants, mountains, rivers, seas, or other living or non-living things in the physical world; or, Physical works or physical models obtained by constructing with paper materials / wood materials / paper materials and glue / wood materials and glue / wood materials, glue and tools. Different forms of the physical works or physical models can also be used to represent different forms of buildings, vehicles, animals, plants, mountains, rivers, seas or other living or non-living things in the physical world.

[0061] In an actual scenario, the user who constructs the physical work can be a teenage user. In this case, after the teenage user completes the construction of the physical work, the guardian user of the teenage user can collect an image of the physical work through a user terminal to obtain a work image. Or, if the teenage user owns a user terminal, the teenage user can also collect an image of the physical work through the user terminal to obtain a work image; In addition, the user who constructs the physical work can be an adult user. After the adult user completes the construction of the physical work, the adult user can collect an image of the physical work through the user terminal to obtain a work image.

[0062] Specifically in implementation, it is obtained by collecting an image of the physical work through a user terminal. After the image collection is completed and the work image is obtained, the user terminal uploads the work image. Correspondingly, the work image of the physical work uploaded by the user terminal is obtained here; Specifically, to collect an image of the physical work to obtain a work image, the image collection can be carried out by accessing the application installed on the user terminal. After the image collection is completed, the user terminal can upload the work image; Or, the image collection can also be carried out by accessing the service or subroutine provided by the application. After the image collection is completed, the user terminal can upload the work image.

[0063] Furthermore, after obtaining the work image of the physical work uploaded by the user terminal, it is necessary to identify the work image and conduct relevant knowledge learning based on the identification. The process of identifying the work image can be achieved by calling a large language model. Specifically, before calling the large language model, an identification prompt word for work identification and object association retrieval of the work image is generated. For example, the generated identification prompt word contains a description text for work identification and also contains a description text for object association retrieval. These two parts together form the identification prompt word (identification prompt text) for work identification and object association retrieval.

[0064] In this embodiment, there is a certain relationship between the physical work and the physical objects that actually exist in the physical world. This relationship can be that the physical work constructed by the user can represent the physical objects that actually exist in the physical world, or the physical work constructed by the user is used to refer to the physical objects that actually exist in the physical world. For example, if the color and structure of the building block model built by the user are similar to those of a certain building in the physical world, then this building block model can represent or refer to the building in the physical world. Another example is that if the wooden car model built by the user is similar to an ambulance in the physical world, then this building block model can represent or refer to the ambulance in the physical world.

[0065] Specifically, when implementing, a large language model is called for work recognition and object association retrieval. The function is to recognize and understand the physical work, and based on the information obtained from the recognition, determine which or which real physical objects in the physical world the physical work represents or refers to. That is, the purpose of work recognition and object association retrieval is to determine the physical objects represented or referred to by the physical work.

[0066] In the specific execution process, when inputting the recognition prompt and the work image into the large language model for work recognition and object association retrieval, in the work recognition link, the work image can be recognized to obtain recognition information. In the object association retrieval link, according to the obtained recognition information, the physical object having a representation relationship with the physical work is retrieved as the physical object represented by the physical work. The large language model refers to a work recognition model that can perform corresponding recognition processing on the input work image according to the input recognition prompt, and this large language model is a multi-modal model that can support text and image input. The large language model can adopt a natural language model with a neural network architecture containing a large number of parameters, and can also adopt a pre-trained large language model (Large Language Model, LLM) or an open-source large language model. Or, it can also be obtained by fine-tuning the base large language model or the large language model.

[0067] Specifically, in the process of work recognition, in order to improve the recognition success rate and recognition accuracy of the work image, the work image blocks can be extracted from the work image. The extracted work image blocks are the image blocks corresponding to the physical work in the work image. It is also possible to recognize the work image from multiple dimensions based on the work image blocks to obtain corresponding recognition information. In an optional implementation manner provided in this embodiment, work recognition includes: Performing shape element recognition on the work image blocks extracted from the work image to obtain shape elements, and performing color relationship analysis on the work image blocks and / or the shape elements to obtain color relationships; Determining the associated physical objects of the historical target physical objects based on the historical target physical objects represented by the historical physical works recorded in the user recognition records.

[0068] In the specific execution process, the work image blocks of the physical work can be extracted from the work image through the image segmentation module or image segmentation algorithm configured by the large language model. The shape elements can also be obtained by identifying the shape elements of the work image blocks through the shape recognition module configured by the large language model. The color relationship can also be obtained by analyzing the color relationship of the work image blocks and / or shape elements through the color relationship module configured by the large language model. In addition, the associated physical object of the historical target physical object represented by the historical physical work recorded in the user recognition record can be determined through the historical retrieval module configured by the large language model. The user recognition record refers to the record of the user's terminal or the user's previous recognition of physical works. The obtained shape elements, color relationships, and associated physical objects are used as recognition information.

[0069] It should be noted that the above-mentioned work recognition is performed on the work image blocks in three dimensions: shape elements, color relationships, and historical recognition objects. In the actual scenario, the work image blocks can also be recognized in any one or any two of the three dimensions of shape elements, color relationships, and historical recognition objects according to actual needs to obtain corresponding recognition information. For example, work recognition includes: identifying the shape elements of the work image blocks extracted from the work image to obtain the shape elements, and analyzing the color relationship of the work image blocks and / or shape elements to obtain the color relationship. The shape elements and color relationships can be used as recognition information. Another example is that work recognition includes identifying the shape elements of the work image blocks extracted from the work image, or work recognition includes analyzing the color relationship of the work image blocks and / or shape elements to obtain the color relationship.

[0070] Based on this, in the process of object association retrieval, on the basis of the recognition information obtained from work recognition, retrieval is performed according to the recognition information. Specifically, the physical objects having a representation relationship with the physical work are retrieved. Specifically, the physical objects can be retrieved according to the shape elements, color relationships, and / or associated objects to obtain the physical objects represented by the physical work.

[0071] In an optional implementation manner provided in this embodiment, the object association retrieval includes: Retrieving candidate physical objects having a representation relationship with the physical work according to the shape elements, color relationships, and / or associated physical objects, and scoring each retrieved candidate physical object; Calculating the retrieval scores of each candidate physical object according to the obtained scores and score weights, and selecting at least one candidate physical object as the physical object according to the retrieval scores.

[0072] During the specific execution process, candidate physical objects having a representational relationship with the physical work are retrieved based on shape elements, color relationships, and / or associated physical objects. For the retrieved candidate physical objects, they can be scored from three dimensions: shape elements, color relationships, and / or associated physical objects. The score can specifically be a probability, which refers to the possibility of evaluating the candidate physical object as a physical object representing the physical work in the three dimensions of shape elements, color relationships, and / or associated physical objects. Or, the score can also be obtained by using a scoring algorithm; Based on the scores of each candidate physical object in these three dimensions and in combination with the respective scoring weights of these three dimensions, the weighted score of each candidate physical object can be obtained through weighted calculation as the retrieval score. After obtaining the retrieval score, the candidate physical objects can be sorted in descending order, and the top N candidate physical objects can be selected as the physical objects representing the physical work.

[0073] In practical applications, in order to improve the accuracy of recognizing and processing the work image, three-dimensional modeling can also be performed on the work image blocks included in the work image, and based on this three-dimensional modeling, work recognition and object association retrieval can be carried out. Specifically, in an optional implementation manner provided in this embodiment, work recognition and object association retrieval are implemented in the following manner: Three-dimensional modeling is performed on the work image blocks included in the work image to obtain a virtual work, and physical recognition is performed on the virtual work to obtain first recognition information; Shape element recognition, color relationship analysis, and / or associated object query are performed on the virtual work to obtain second recognition information; Based on the first recognition information, the second recognition information, and the virtual work, physical objects having a representational relationship with the physical work are retrieved.

[0074] Among them, during the process of performing three-dimensional modeling on the work image blocks to obtain a virtual work, the three-dimensional modeling module configured by the large language model can be used to perform three-dimensional modeling on the work image blocks to obtain a virtual work, and the physical recognition module configured by the large language model can be used to perform physical recognition on the virtual work to obtain first recognition information.

[0075] Step S804, receive and display the physical objects returned by the server, and submit the selected target physical objects in the physical objects to the server.

[0076] After uploading the work image of the physical work to the server, and the server obtains the physical object representing the physical work through work recognition and object association retrieval by the large language model, the server returns the obtained physical object to the user terminal. Specifically, in the process of returning the physical object, based on obtaining the object identifier and object image of the physical object, the object identifier and object image of the physical object can be returned. Here, receive and display the physical object returned by the server. On this basis, selection can be made among the returned physical objects. Specifically, select the physical object that truly represents the current physical work among the physical objects. The physical object that truly represents the selected physical work is called the target physical object. Specifically, after completing the selection of the target physical object, submit the selected target physical object to the server to perform knowledge learning interaction of the target physical object in cooperation with the server.

[0077] For example, after a teenage user builds a block model in the shape of a spire with building blocks, the parent user collects the model image of the block model through a subroutine of the application installed on the user terminal, as Figure 3 shown; after completing the collection of the model image of the block model, the user terminal uploads the model image to the server in the background of the subroutine. After receiving the model image, the server performs work recognition and object association retrieval on the model image, and obtains 3 physical objects: the Eiffel Tower, the Tokyo Tower, and the TV Tower, and returns the names and images of these 3 physical objects to the user terminal. The user terminal receives and displays the names and images of these 3 physical objects returned by the server, as Figure 4 shown; Regarding the names and images of the 3 physical objects displayed, the teenage user or the parent user can select the physical object that truly represents the block model built with building blocks among these 3 physical objects. If the teenage user builds the block model based on the Eiffel Tower as the prototype or wants to build a block model of the Eiffel Tower, then the physical object of the Eiffel Tower among the 3 physical objects can be selected, and the selected physical object is submitted to the server.

[0078] Step S806, perform interactive display of the knowledge data according to the knowledge data of the target physical object returned by the server.

[0079] After submitting the target physical object to the server, after the server obtains the submitted target physical object, that is, after obtaining the target physical object that truly represents the physical work, it generates the knowledge data of the target physical object and returns it. Correspondingly, here, according to the knowledge data of the target physical object returned by the server, interactive display of the knowledge data is performed. Specifically, the interactive display of the knowledge data can be a knowledge learning page that displays the knowledge data of the target physical object, so that the user who constructs the physical work can learn the knowledge data of the target physical object and also learn relevant knowledge during the construction process of the physical work.

[0080] The above steps S802 to S806 provided in this embodiment can be executed by the user terminal. It should be noted that the above steps S802 to S806 executed by the user terminal can cooperate with the steps S202 to S208 provided in the above method embodiment during the execution process. Therefore, when reading this embodiment, please refer to the corresponding content of the steps S202 to S208 provided in the above method embodiment, and when reading the above method embodiment, please refer to the corresponding content of the steps S802 to S806 provided in this embodiment.

[0081] The following takes the application of a work recognition processing method provided in this embodiment in a work recognition service scenario as an example, in combination with Figure 7 , to further illustrate the work recognition processing method provided in this embodiment. See Figure 7 , the work recognition processing method applied to the work recognition service scenario specifically includes the following steps.

[0082] Step S702, collect the work image of the physical work by accessing the work recognition service and upload it to the server.

[0083] Step S712, receive and display multiple physical objects representing the physical work returned by the server.

[0084] Step S714, submit the selected target physical object among the multiple physical objects to the server.

[0085] Step S720, receive the learning knowledge text of the target physical object returned by the server.

[0086] Step S722, perform interactive display of the learning knowledge text on the service page.

[0087] It should be noted that any one step or any combination of steps among step S702, steps S712 to S714, and steps S720 to S722 can be combined with any one step or any combination of steps among the above steps S802 to S806 according to the needs of implementation and deployment to form a new implementation manner; in addition, according to the actual deployment needs, any one or any combination of technical features among step S702, steps S712 to S714, and steps S720 to S722 can be selected and combined with any one or more technical features provided by the above steps S802 to S806 to form a new implementation manner; or, any one or any combination of technical features among step S702, steps S712 to S714, and steps S720 to S722 can also be replaced by any one or more technical feature combinations provided by the above steps S802 to S806 according to the actual deployment needs to form a new implementation manner, which will not be elaborated here one by one.

[0088] In addition, it should also be noted that the above steps S702, steps S712 to S714, and steps S720 to S722 provided in this embodiment can be executed by the user terminal. It should be noted that the above steps S702, steps S712 to S714, and steps S720 to S722 executed by the user terminal can cooperate with the steps S704, steps S706 to S710, and steps S716 to S718 executed by the server in the above embodiment during the execution process. Therefore, when reading this embodiment, please refer to the corresponding content of steps S704, steps S706 to S710, and steps S716 to S718 provided in the above method embodiment, and when reading the above method embodiment, please refer to the corresponding content of steps S702, steps S712 to S714, and steps S720 to S722 provided in this embodiment.

[0089] An embodiment of a work recognition processing device provided in this specification is as follows: In the above embodiment, a work recognition processing method is provided. Correspondingly, a work recognition processing device is also provided. The following will be described with reference to the drawings.

[0090] Refer to Figure 9 , which shows a schematic diagram of an embodiment of a work recognition processing device provided in this embodiment.

[0091] Since the device embodiment corresponds to the method embodiment, the description is relatively simple. For the relevant parts, please refer to the corresponding description of the method embodiment provided above. The device embodiments described below are merely illustrative.

[0092] This embodiment provides a work recognition processing device, which includes: A prompt word generation module 902, configured to obtain a work image of a physical work uploaded by a user terminal and generate an identification prompt word for performing work recognition and object association retrieval on the work image; An identification and retrieval module 904, configured to input the identification prompt word and the work image into a large language model for work recognition and object association retrieval to obtain a physical object represented by the physical work; A physical object return module 906, configured to return the physical object to the user terminal and receive a target physical object selected by the user terminal from the physical object; A knowledge generation module 908, configured to generate knowledge data of the target physical object and return it to the user terminal.

[0093] Another embodiment of the work recognition processing device provided in this specification is as follows: In the above embodiment, another work recognition processing method is provided. Correspondingly, another work recognition processing device is also provided, which will be described below with reference to the accompanying drawings.

[0094] Refer to Figure 10 , which shows a schematic diagram of an embodiment of a work recognition processing device provided in this embodiment.

[0095] Since the device embodiment corresponds to the method embodiment, the description is relatively simple. For related parts, please refer to the corresponding description of the method embodiment provided above. The device embodiments described below are merely illustrative.

[0096] This embodiment provides a work recognition processing device, which includes: A work image acquisition module 1002, configured to acquire a work image of a physical work and upload it to a server to call a large language model according to the work image and an identification prompt word for work recognition and object association retrieval to obtain a physical object; A physical object selection module 1004, configured to receive and display the physical object returned by the server and submit the selected target physical object in the physical object to the server; A knowledge data display module 1006, configured to perform interactive display of the knowledge data according to the knowledge data of the target physical object returned by the server.

[0097] An embodiment of a work recognition processing device provided in this specification is as follows: Corresponding to the above-described method for processing work recognition, based on the same technical concept, one or more embodiments of this specification also provide a work recognition processing device, which is used to execute the above-provided method for processing work recognition. Figure 11 It is a schematic structural diagram of a work recognition processing device provided by one or more embodiments of this specification.

[0098] A work recognition processing device provided in this embodiment includes: As Figure 11 shown, the work recognition processing device may vary greatly due to configuration or performance, and may include one or more processors 1101 and a memory 1102. One or more storage applications or data may be stored in the memory 1102. Among them, the memory 1102 may be short-term storage or persistent storage. The application programs stored in the memory 1102 may include one or more modules (not shown in the figure), and each module may include a series of computer-executable instructions in the work recognition processing device. Further, the processor 1101 may be set to communicate with the memory 1102 and execute a series of computer-executable instructions in the memory 1102 on the work recognition processing device. The work recognition processing device may also include one or more power supplies 1103, one or more wired or wireless network interfaces 1104, one or more input / output interfaces 1105, etc.

[0099] In a specific embodiment, the work recognition processing device includes a memory and one or more programs, where one or more programs are stored in the memory, and one or more programs may include one or more modules, and each module may include a series of computer-executable instructions in the work recognition processing device, and is configured to be executed by one or more processors. The one or more programs include the following computer-executable instructions: Obtain the work image of the physical work uploaded by the user terminal and generate an identification prompt word for performing work recognition and object association retrieval on the work image; Input the identification prompt word and the work image into a large language model for work recognition and object association retrieval to obtain the physical object represented by the physical work; Return the physical object to the user terminal and receive the target physical object selected by the user terminal in the physical object; Generate knowledge data of the target physical object and return it to the user terminal.

[0100] Another embodiment of the work recognition processing device provided in this specification is as follows: Another method for identifying works described above, based on the same technical concept, one or more embodiments of this specification also provide another device for identifying works. This device for identifying works is used to execute the method for identifying works provided above. Figure 12 It is a schematic structural diagram of another device for identifying works provided by one or more embodiments of this specification.

[0101] A device for identifying works provided in this embodiment includes: As Figure 12 shown, the device for identifying works can vary greatly due to configuration or performance differences. It can include one or more processors 1201 and a memory 1202. One or more applications or data can be stored in the memory 1202. Among them, the memory 1202 can be short-term storage or persistent storage. The applications stored in the memory 1202 can include one or more modules (not shown in the figure). Each module can include a series of computer-executable instructions in the device for identifying works. Further, the processor 1201 can be set to communicate with the memory 1202 and execute a series of computer-executable instructions in the memory 1202 on the device for identifying works. The device for identifying works can also include one or more power supplies 1203, one or more wired or wireless network interfaces 1204, one or more input / output interfaces 1205, one or more keyboards 1206, etc.

[0102] In a specific embodiment, the device for identifying works includes a memory and one or more programs. One or more of the programs are stored in the memory, and one or more of the programs can include one or more modules. Each module can include a series of computer-executable instructions in the device for identifying works and is configured to be executed by one or more processors. The one or more programs include the following computer-executable instructions: Collect the work image of the physical work and upload it to the server to call a large language model for work identification and object association retrieval based on the work image and recognition prompt words to obtain physical objects; Receive and display the physical objects returned by the server, and submit the selected target physical objects in the physical objects to the server; According to the knowledge data of the target physical object returned by the server, perform interactive display of the knowledge data.

[0103] An embodiment of a computer-readable storage medium provided in this specification is as follows: Corresponding to a method for identifying and processing works described above, based on the same inventive concept, one or more embodiments of this specification also provide a computer-readable storage medium.

[0104] The computer-readable storage medium provided in this embodiment is used to store computer-executable instructions, and when the computer-executable instructions are executed, the following processes are implemented: Obtain the work image of the physical work uploaded by the user terminal, and generate an identification prompt word for performing work identification and object association retrieval on the work image; Input the identification prompt word and the work image into a large language model for work identification and object association retrieval to obtain the physical object represented by the physical work; Return the physical object to the user terminal, and receive the target physical object selected by the user terminal from the physical objects; Generate the knowledge data of the target physical object and return it to the user terminal.

[0105] It should be noted that the embodiment of the computer-readable storage medium in this specification and the embodiment of the method for identifying and processing works in this specification are based on the same inventive concept. Therefore, the specific implementation of this embodiment can refer to the implementation of the corresponding method described above, and the repeated parts will not be elaborated.

[0106] Another embodiment of the computer-readable storage medium provided in this specification is as follows: Corresponding to another method for identifying and processing works described above, based on the same inventive concept, one or more embodiments of this specification also provide another computer-readable storage medium.

[0107] The computer-readable storage medium provided in this embodiment is used to store computer-executable instructions, and when the computer-executable instructions are executed, the following processes are implemented: Collect the work image of the physical work and upload it to the server to call a large language model for work identification and object association retrieval based on the work image and the identification prompt word to obtain a physical object; Receive and display the physical object returned by the server, and submit the selected target physical object in the physical object to the server; Perform interactive display of the knowledge data according to the knowledge data of the target physical object returned by the server.

[0108] It should be noted that the embodiment of the another computer-readable storage medium in this specification and the embodiment of the another method for identifying and processing works in this specification are based on the same inventive concept. Therefore, the specific implementation of this embodiment can refer to the implementation of the corresponding method described above, and the repeated parts will not be elaborated.

[0109] An embodiment of a computer program product provided in this specification is as follows: Corresponding to the described method for processing work recognition, based on the same inventive concept, one or more embodiments of this specification also provide a computer program product.

[0110] A computer program product includes computer programs / instructions, and when the computer programs / instructions are executed by a processor, the following steps are implemented: Obtain a work image of a physical work uploaded by a user terminal, and generate an identification prompt word for performing work recognition and object association retrieval on the work image; Input the identification prompt word and the work image into a large language model for work recognition and object association retrieval to obtain the physical object represented by the physical work; Return the physical object to the user terminal, and receive the target physical object selected by the user terminal from the physical objects; Generate knowledge data of the target physical object and return it to the user terminal.

[0111] It should be noted that the embodiment of a computer program product in this specification and the embodiment of a method for processing work recognition in this specification are based on the same inventive concept. Therefore, the specific implementation of this embodiment can refer to the implementation of the corresponding method described above, and the repeated parts will not be elaborated.

[0112] Another embodiment of a computer program product provided in this specification is as follows: Corresponding to the described another method for processing work recognition, based on the same inventive concept, one or more embodiments of this specification also provide another computer program product.

[0113] A computer program product includes computer programs / instructions, and when the computer programs / instructions are executed by a processor, the following steps are implemented: Collect a work image of a physical work and upload it to a server to call a large language model for work recognition and object association retrieval based on the work image and an identification prompt word to obtain a physical object; Receive and display the physical object returned by the server, and submit the selected target physical object in the physical object to the server; Perform an interactive display of the knowledge data according to the knowledge data of the target physical object returned by the server.

[0114] It should be noted that the embodiment of another computer program product in this specification and the embodiment of another method for processing work recognition in this specification are based on the same inventive concept. Therefore, the specific implementation of this embodiment can refer to the implementation of the corresponding method described above, and the repeated parts will not be elaborated.

[0115] Each embodiment in this specification is described in a progressive manner. For the same or similar parts among the embodiments, reference can be made to each other. The key point of each embodiment is to illustrate the differences from other embodiments. For example, the device embodiment, the equipment embodiment, the computer-readable storage medium embodiment, and the computer program product embodiment are all similar to the method embodiment, so the description is relatively simple. To read the relevant content in the device embodiment, the equipment embodiment, the computer-readable storage medium embodiment, and the computer program product embodiment, please refer to the corresponding part of the method embodiment for description.

[0116] The above describes specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in a different order from that in the embodiments and still achieve the desired result. Additionally, the processes depicted in the drawings do not necessarily require the specific order or consecutive order shown to achieve the desired result. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0117] In the 1930s, it was obvious to distinguish whether an improvement to a technology was a hardware improvement (e.g., improvement to circuit structures such as diodes, transistors, switches, etc.) or a software improvement (improvement to method processes). However, with the development of technology, many improvements to method processes today can be regarded as direct improvements to hardware circuit structures. Almost all designers obtain the corresponding hardware circuit structures by programming the improved method processes into the hardware circuits. Therefore, it cannot be said that an improvement to a method process cannot be implemented with hardware entity modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logical function is determined by the user programming the device. Designers can program themselves to "integrate" a digital system on a piece of PLD without having to ask a chip manufacturer to design and fabricate a dedicated integrated circuit chip. Moreover, nowadays, instead of manually fabricating integrated circuit chips, this programming is mostly implemented using "logic compiler" software, which is similar to the software compiler used in program development and writing. The original code before compilation also has to be written in a specific programming language, which is called a Hardware Description Language (HDL). There is not only one type of HDL, but many types, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc. The most commonly used ones currently are VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also be aware that by simply performing a little logical programming on the method process with the above-mentioned several hardware description languages and programming it into the integrated circuit, it is easy to obtain the hardware circuit that implements the logical method process.

[0118] The controller can be implemented in any suitable manner. For example, the controller can take the form of, for example, a microprocessor or a processor and a computer-readable medium storing computer-readable program code (such as software or firmware) executable by the (micro)processor, logic gates, switches, an application specific integrated circuit (ASIC), a programmable logic controller, and an embedded microcontroller. Examples of the controller include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicone Labs C8051F320. The memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art also know that, in addition to implementing the controller in the form of pure computer-readable program code, it is entirely possible to logically program the method steps to enable the controller to be implemented in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers, and embedded microcontrollers, etc. to achieve the same functions. Therefore, such a controller can be considered a hardware component, and the devices included therein for implementing various functions can also be regarded as the structures within the hardware component. Or even, the devices for implementing various functions can be regarded as either software modules for implementing the method or structures within the hardware component.

[0119] The systems, devices, modules, or units illustrated in the above embodiments can be specifically implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer can be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or any combination of these devices.

[0120] For the convenience of description, when describing the above devices, they are described separately as various units according to their functions. Of course, when implementing the embodiments of this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0121] Those skilled in the art should understand that one or more embodiments of this specification can be provided as a method, a system, or a computer program product. Therefore, one or more embodiments of this specification can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, this specification can take the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk memories, CD-ROMs, optical memories, etc.) containing computer-usable program code.

[0122] This specification is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the specification. It should be understood that each flow and / or block in the flowchart and / or block diagram, and the combination of flows and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in one or more flows Figure 1 or more flows and / or blocks Figure 1 or more blocks.

[0123] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including instruction means, and the instruction means implement the functions specified in one or more flows Figure 1 or more flows and / or blocks Figure 1 or more blocks.

[0124] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one or more flows Figure 1 or more flows and / or blocks Figure 1 or more blocks.

[0125] In a typical configuration, a computing device includes one or more processors (CPUs), an input / output interface, a network interface, and memory.

[0126] The memory may include non-permanent memory in the form of computer-readable media, random access memory (RAM), and / or non-volatile memory, such as read-only memory (ROM) or flash memory (flash RAM). The memory is an example of computer-readable media.

[0127] A computer-readable medium includes permanent and non-permanent, removable and non-removable media that can implement information storage by any method or technology. The information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer-readable storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette tapes, disk storage or other magnetic storage devices, or any other non-transitory medium that can be used to store information that can be accessed by a computing device. As defined herein, a computer-readable medium does not include transitory computer-readable media, such as modulated data signals and carrier waves.

[0128] It should also be noted that the term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, such that a process, method, article or apparatus comprising a series of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article or apparatus. Without further limitation, an element defined by the statement "comprising at least one..." does not exclude the presence of additional identical elements in the process, method, article or apparatus comprising the element.

[0129] One or more embodiments of this specification may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. One or more embodiments of this specification may also be practiced in a distributed computing environment where tasks are performed by remote processing devices connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media including storage devices.

[0130] The above are only embodiments of this document and are not used to limit this document. For those skilled in the art, this document may have various changes and modifications. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of this document shall be included within the scope of the claims of this document.

Claims

1. A method for identifying and processing works, comprising: Obtaining a work image of a physical work uploaded by a user terminal, and generating an identification prompt word for performing work identification and object association retrieval on the work image; Inputting the identification prompt word and the work image into a large language model for work identification and object association retrieval to obtain a physical object represented by the physical work; Returning the physical object to the user terminal, and receiving a target physical object selected by the user terminal from the physical objects; Generating knowledge data of the target physical object and returning it to the user terminal.

2. The method for identifying and processing works according to claim 1, wherein the work identification includes: Performing shape element identification on a work image block extracted from the work image to obtain shape elements, and performing color relationship analysis on the work image block and / or the shape elements to obtain color relationships; Determining an associated physical object of the historical physical object according to the historical physical object represented by the historical physical work recorded in the user identification record.

3. The method for identifying and processing works according to claim 2, wherein the object association retrieval includes: Retrieving candidate physical objects having a representation relationship with the physical work according to the shape elements, the color relationships, and the associated physical objects, and scoring each of the retrieved candidate physical objects; Calculating a retrieval score of each candidate physical object according to the obtained scores and score weights, and selecting at least one candidate physical object as the physical object according to the retrieval score.

4. The method for identifying and processing works according to claim 1, wherein generating the knowledge data of the target physical object includes: Generating a learning knowledge text of the target physical object, and performing voice conversion on the learning knowledge text to obtain learning knowledge voice.

5. The method for identifying and processing works according to claim 4, wherein the physical work is constructed by a teenage user, and the work image is uploaded through the user terminal of the guardian user of the teenage user; The learning knowledge text includes: Learning knowledge data or teaching knowledge data provided to the teenage user.

6. The method for processing work identification according to claim 1, wherein the physical work includes: A physical model obtained by physically constructing according to articles, tools, and / or materials; Wherein the physical construction includes physical model building, physical model combination, and / or physical model construction.

7. The method for identifying and processing works according to claim 1, wherein performing work identification and object association retrieval is implemented in the following manner: Performing three-dimensional modeling on the basis of the work image blocks included in the work image to obtain a virtual work, and performing physical identification on the virtual work to obtain first identification information; Performing shape element identification, color relationship analysis, and / or associated object query on the virtual work to obtain second identification information; Retrieving a physical object having a representation relationship with the physical work according to the first identification information, the second identification information, and the virtual work.

8. After the step of generating the knowledge data of the target physical object and returning it to the user terminal according to claim 1, further comprising: If an archiving instruction for the work image is detected, a work archive image is generated based on the work image and the object image of the target physical object and stored.

9. The work recognition processing method according to claim 1, after the step of generating the knowledge data of the target physical object and returning it to the user terminal, further includes: If a sharing instruction for the work image is detected, a work sharing message is generated and sent according to the selected sharing channel; According to the comment information submitted after the work sharing message is triggered, the comment information is associated with the work image or the work archive image.

10. The work recognition processing method according to claim 1, further includes: If it is detected that the work archive images of the same work category in the user storage space meet the task trigger condition, interaction data is generated based on the interaction materials and sent to the user terminal; Wherein, the task trigger condition includes: the positional relationship of the geographical regions of the buildings corresponding to the building model works under the building model classification meets a preset positional relationship.

11. A work recognition processing method, including: Collecting a work image of a physical work and uploading it to a server, and calling a large language model according to the work image and recognition prompt words to perform work recognition and object association retrieval to obtain a physical object representing the physical work; Receiving and displaying the physical object returned by the server, and submitting the selected target physical object in the physical object to the server; Performing interactive display of the knowledge data according to the knowledge data of the target physical object returned by the server.

12. The work recognition processing method according to claim 11, wherein the work recognition includes: Performing shape element recognition on the work image blocks extracted from the work image to obtain shape elements, and performing color relationship analysis on the work image blocks and / or the shape elements to obtain color relationships; Determining the associated physical object of the historical physical object according to the historical physical object representing the historical physical work recorded in the user recognition record.

13. The object association retrieval according to claim 12 includes: Retrieving candidate physical objects having a representation relationship with the physical work according to the shape elements, the color relationships, and the associated physical objects, and scoring each of the retrieved candidate physical objects; Calculating the retrieval scores of the candidate physical objects according to the obtained scores and score weights, and selecting at least one candidate physical object as the physical object according to the retrieval scores.

14. The work recognition processing method according to claim 11, wherein the physical work is constructed by a teenage user, and the work image is uploaded through the user terminal of the guardian user of the teenage user; The knowledge data includes learning knowledge data or teaching knowledge data provided to the teenage user.

15. The method for processing work identification according to claim 11, wherein the physical work includes: A physical model obtained by physically constructing according to items, tools, and / or materials; Wherein, the physical construction includes physical model building, physical model combination, and / or physical model construction.

16. A work recognition processing device, including: A prompt word generation module, configured to obtain a work image of a physical work uploaded by a user terminal and generate an identification prompt word for performing work identification and object association retrieval on the work image; An identification and retrieval module, configured to input the identification prompt word and the work image into a large language model for work identification and object association retrieval to obtain a physical object represented by the physical work; A physical object return module, configured to return the physical object to the user terminal and receive a target physical object selected by the user terminal from the physical objects; A knowledge generation module, configured to generate knowledge data of the target physical object and return it to the user terminal.

17. A work identification processing device, comprising: A work image acquisition module, configured to acquire a work image of a physical work and upload it to a server, so as to call a large language model according to the work image and an identification prompt word for work identification and object association retrieval to obtain a physical object; A physical object selection module, configured to receive and display the physical object returned by the server and submit the selected target physical object in the physical objects to the server; A knowledge data display module, configured to perform interactive display of the knowledge data according to the knowledge data of the target physical object returned by the server.

18. A work identification processing device, comprising: A processor; And a memory configured to store computer-executable instructions, the computer-executable instructions, when executed, cause the processor to: Obtain a work image of a physical work uploaded by a user terminal and generate an identification prompt word for performing work identification and object association retrieval on the work image; Input the identification prompt word and the work image into a large language model for work identification and object association retrieval to obtain a physical object represented by the physical work; Return the physical object to the user terminal and receive a target physical object selected by the user terminal from the physical objects; Generate knowledge data of the target physical object and return it to the user terminal.

19. A work identification processing device, comprising: A processor; And a memory configured to store computer-executable instructions, the computer-executable instructions, when executed, cause the processor to: Acquire a work image of a physical work and upload it to a server, so as to call a large language model according to the work image and an identification prompt word for work identification and object association retrieval to obtain a physical object; Receive and display the physical object returned by the server and submit the selected target physical object in the physical objects to the server; Perform interactive display of the knowledge data according to the knowledge data of the target physical object returned by the server.

20. A computer-readable storage medium for storing computer-executable instructions, the computer-executable instructions, when executed, implement the steps of the method according to claim 1 or 11.

Images