Method, apparatus, electronic device, storage medium and computer program for specifying meaning of mark

Abstract

To provide a method, an apparatus, an electronic device, a storage medium, and a computer program product for specifying the meaning of a mark.SOLUTION: The method includes steps of determining whether or not a mark object is included in a target image (201), and specifying a meaning represented by the mark object according to the determination that the mark object is included in the target image (202).SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present application relates to the field of computer technology, specifically to the field of image recognition and artificial intelligence, and in particular to a method and device, electronic device, storage medium and computer program for identifying the meaning of a sign, which can be applied to the scene of recognizing the meaning of a sign. [Background technology]

[0002] In daily life, people can see various signs, such as laundry instructions and traffic signs. When people see a sign they do not know, they have a need to know the meaning of the sign. However, the products currently on the market do not support sign recognition. Summary of the Invention [Means for solving the problem]

[0003] The present disclosure provides a method, an apparatus, an electronic device, a storage medium, and a computer program for determining the meaning of a sign.

[0004] According to a first aspect, there is provided a method for identifying the meaning of a sign, the method including the steps of determining whether or not a target image contains a sign object, and identifying the meaning represented by the sign object in response to determining that the target image contains the sign object.

[0005] According to a second aspect, there is provided an apparatus for identifying a meaning of a sign, the apparatus comprising: a sign identification unit configured to determine whether or not a target image includes a sign object; and a meaning identification unit configured to identify a meaning represented by the sign object in response to determining that the target image includes the sign object.

[0006] According to a third aspect, there is provided an electronic device comprising at least one processor and a memory communicatively connected to the at least one processor, the memory storing instructions executable by the at least one processor, the instructions, when executed by the at least one processor, causing the at least one processor to perform a method according to any of the embodiments of the first aspect.

[0007] According to a fourth aspect, there is provided a non-transitory computer readable storage medium having computer instructions stored thereon, the computer instructions causing a computer to perform a method according to any of the embodiments of the first aspect.

[0008] According to a fifth aspect, there is provided a computer program which, when executed by a processor, implements a method according to any embodiment of the first aspect.

[0009] According to the technology disclosed herein, a method and device for identifying the meaning of a sign are provided, which automatically recognize whether or not a sign object is included in a target image, and if the target image includes a sign object, can automatically identify the meaning expressed by the sign object. The signs efficiently expressed by the user through the target image improve the efficiency of acquiring sign data and the efficiency of identifying the sign meaning.

[0010] It should be noted that the contents described in the summary of the invention are not intended to limit the key features or important features of the embodiments of the present disclosure, and are not intended to limit the scope of the present disclosure. Other features of the present disclosure will become easier to understand from the following description. The drawings are used for a better understanding of the present disclosure and are not intended to be limitations to the present disclosure. [Brief description of the drawings]

[0011] [Figure 1] FIG. 1 illustrates an exemplary system architecture to which an embodiment of the present disclosure can be applied. [Diagram 2]1 is a flow chart illustrating one embodiment of a method for determining the meaning of a sign according to the present disclosure. [Diagram 3] 13A and 13B are schematic diagrams of a plurality of types of sub-label objects in a laundry indication. [Figure 4] FIG. 2 is a schematic diagram illustrating an application scene of the method for identifying the meaning of a sign according to an embodiment of the present disclosure. [Diagram 5] 10 is a flow chart illustrating another embodiment of a method for determining the meaning of a sign according to the present disclosure. [Figure 6] FIG. 1 is a structural diagram of one embodiment of an apparatus for determining the meaning of a sign according to the present disclosure. [Figure 7] FIG. 1 is a schematic structural diagram of a computer system suitable for implementing embodiments of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] The following describes exemplary embodiments of the present disclosure with reference to the drawings, and various details of the embodiments of the present disclosure are described to facilitate understanding, but it should be understood that these are merely illustrative. Therefore, it should be understood that those skilled in the art can make various changes and modifications to the embodiments of the present disclosure without departing from the scope and spirit of the present disclosure. In the following description, for clarity and simplicity, the description of known functions and structures is omitted.

[0013] In the technical solution disclosed herein, the collection, storage, use, processing, transmission, provision, disclosure and other processing of relevant user personal information shall all comply with the provisions of relevant laws and regulations and shall not violate public order and morals.

[0014] FIG. 1 illustrates an exemplary system architecture 100 in which the method and apparatus for determining the meaning of a sign according to the present disclosure can be applied.

[0015] 1, system architecture 100 may include terminal devices 101, 102, 103, a network 104, and a server 105. Terminal devices 101, 102, 103 are communicatively connected to form a topology network, and network 104 is used as a medium to provide a communication link between terminal devices 101, 102, 103 and server 105. Network 104 may include various types of connections, such as wired, wireless communication links, or fiber optic cables.

[0016] The terminal devices 101, 102, and 103 may be hardware devices or software that support network connection for data exchange and data processing. If the terminal devices 101, 102, and 103 are hardware, they may be various electronic devices that support functions such as network connection, information acquisition, interaction, display, and processing, including, but not limited to, smartphones, tablet computers, e-book readers, laptop computers, and desktop computers. If the terminal devices 101, 102, and 103 are software, they may be installed in the electronic devices exemplified above. It may be implemented, for example, as multiple software or software modules for providing distributed services, or may be implemented as a single software or software module. There is no particular limitation here.

[0017] The server 105 may be a server that provides various services, for example, a back-processing server that acquires target images uploaded by users via the terminal devices 101, 102, 103 and identifies the meanings of sign objects included in the target images. As an example, the server 105 may be a cloud server.

[0018] The server may be hardware or software. If the server is hardware, it may be implemented as a distributed server cluster consisting of multiple servers, or as a single server. If the server is software, it may be implemented as multiple pieces of software or software modules (e.g., software or software modules for providing distributed services), or as a single piece of software or software module. There are no particular limitations here.

[0019] In addition, the method for identifying the meaning of a sign according to the embodiment of the present disclosure may be executed by a server, may be executed by a terminal device, or may be executed by cooperation between the server and the terminal device. Correspondingly, each part (e.g., each unit) included in the device for identifying the meaning of a sign may be provided entirely in the server, may be provided entirely in the terminal device, or may be provided individually in the server and the terminal device.

[0020] It should be understood that the number of terminal devices, networks, and servers in Fig. 1 is merely exemplary. The number of terminal devices, networks, and servers may be arbitrarily increased or decreased according to the needs of implementation. If the electronic device on which the method for identifying the meaning of a sign operates does not need to perform data transmission with other electronic devices, the system architecture may only include the electronic device on which the method for identifying the meaning of a sign operates (e.g., a terminal device or a server).

[0021] Please refer to Figure 2. Figure 2 is a flow chart of a method for determining the meaning of a sign according to an embodiment of the present disclosure. The flow 200 includes the following steps:

[0022] In step 201, it is determined whether or not a marker object is included in the target image.

[0023] In this embodiment, an entity executing the method for identifying the meaning of a sign (e.g., the server shown in FIG. 1) may obtain a target image remotely or locally via a wired or wireless network connection, and determine whether the target image includes a sign object.

[0024] The sign represented by the sign object may be a sign in the narrow sense or a sign in the broad sense. In the narrow sense, it is various signs commonly seen in daily life, such as laundry instructions, traffic signs, danger signs, and power signs. In the broad sense, it is a sign such as a brand or a flower.

[0025] For example, the execution entity may determine whether a target image includes a sign object by a pre-trained sign object detection model, where the sign object detection model is used to perform a classification operation on the target image to distinguish between images including a sign object and images not including a sign object.

[0026] The label object detection model may be a one-stage target detection model or a two-stage target detection model. In a one-stage target detection model, first, features are extracted from the input image by a convolutional neural network (CNN), and then these features are transmitted to a prediction head to predict the target bounding box at each position and its category confidence. Finally, overlapping bounding boxes are merged and filtered using techniques such as non-maximum suppression (NMS) to obtain the final detection result. For example, there are models such as SSD (Single Shot Multibox Detector), YOLO (You Only Look Once, an object detection algorithm that simultaneously identifies the type and location of an object with a single image recognition), and RetinaNet (retina network).

[0027] In a two-stage target detection model, first, a convolutional neural network extracts image features and generates a set of candidate regions using a region proposal network (RPN). Next, these candidate regions are converted into fixed-size profiles, and a classifier and regressor are used to perform object classification and high-precision localization for each candidate region. After that, Mask R-CNN (Mask Region-based Convolutional Neural Network) can also add an additional branch to predict the mask for each target instance on top of the above process. Finally, techniques such as non-maximal suppression (NMS) are used to merge overlapping bounding boxes to obtain the final detection result. For example, there are models such as Faster R-CNN (Faster Region-based Convolutional Neural Network), R-CNN (Region-based Convolutional Neural Network), and Mask R-CNN.

[0028] In step 202, in response to determining that the target image contains a sign object, the meaning expressed by the sign object is identified.

[0029] In this embodiment, in response to determining that a marker object is included in a target image, the execution subject can identify the meaning expressed by the marker object.

[0030] As an example, in response to determining that a sign object is included in the target image, the executing entity may use the sign recognition model to identify which specific category of sign object the sign object belongs to, i.e., to identify target category information of the sign object, and further identify the meaning corresponding to the target category information in the sign meaning table as the meaning expressed by the sign object.

[0031] The sign meaning table contains various signs and their corresponding semantic information, and may be generated by statistics and reduction of data.

[0032] As another example, the execution entity may identify the meaning represented by the sign object through a pre-trained meaning identification model, where the meaning identification model is used to represent the correspondence between the target image and the meaning of the sign object in the target image.

[0033] In some optional implementations of this embodiment, the execution entity may execute step 202 as follows.

[0034] In the first step, in response to determining that a sign object is included in a target image, a category to which the sign object belongs is identified.

[0035] In this embodiment, the markings may be classified according to common classification criteria, or may be classified according to preset classification criteria according to actual situations. For example, the marking objects may be divided into categories such as laundry markings, traffic marks, danger marks, and power marks according to common classification criteria.

[0036] As an example, in response to determining that a target image contains a sign object, the executing entity may perform feature extraction on the target image to obtain feature data, calculate the similarity between the feature data and each category feature in the category feature set, and identify the category corresponding to the category feature with the maximum similarity as the category to which the sign object belongs, where each category feature in the category feature set represents one category.

[0037] In the second step, multiple types of sub-sign objects that belong to a category and are included in the target image are identified.

[0038] The target image often includes multiple sub-sign objects of different types in the same category. Next, refer to Fig. 3. Fig. 3 shows multiple sub-sign objects such as "Do not bleach" and "Do not wash" that belong to a category called "washing instructions".

[0039] As an example, the executing entity divides the target image into sub-signal objects of each type to obtain images of the sub-signal objects, performs feature extraction on the images of each sub-signal object to obtain sub-signal feature data, calculates the similarity between each piece of sub-signal feature data and each sub-signal object reference feature in the sub-signal feature set corresponding to the category, and identifies the sub-signal object to which the sub-signal object reference feature corresponding to the maximum similarity belongs as one of the multiple types of sub-signal objects included in the target image.

[0040] Here, the sub-marker feature set corresponding to a category includes sub-marker object reference features of various sub-marker objects belonging to the category.

[0041] As another example, the executing entity may input a target image into a sign recognition model to identify multiple types of sub-sign objects belonging to a category that are included in the target image, where the sign recognition model is, for example, the one-stage target detection model or the two-stage target detection model.

[0042] In the third step, the meaning expressed by each of the multiple types of sub-sign objects is identified.

[0043] For each type of sub-sign object of multiple types, the executing entity may identify the meaning of the sub-sign object of that type by combining the sign recognition model and the sign meaning table, or may identify the meaning represented by the sub-sign object of that type by the meaning identification model.

[0044] In this embodiment, the meanings of multiple sub-sign objects of different types in the same category can be simultaneously identified, improving the accuracy of the sign meaning identification process and the user's information acquisition efficiency.

[0045] In some optional implementations of this embodiment, the execution entity may perform the third step to determine, for each type of sub-marker object among multiple types of sub-marker objects, the meaning represented by that type of sub-marker object based on the number of sub-marker objects of that type in the target image.

[0046] For example, in the case where the sign object is a flower, the meaning of the flower (floral language) varies depending on the type and number of flowers.

[0047] In this embodiment, for each type of sub-marker object among the multiple types, the execution entity first identifies the number of sub-marker objects of that type in the target image, and further identifies the meaning represented by the sub-marker object based on the category and number corresponding to the sub-marker object.

[0048] In this embodiment, the accuracy of the identified meaning is improved by identifying the meaning represented by the sub-sign object based on the category and the number of the sub-sign objects corresponding to the sub-sign object.

[0049] Continuing with the example of the sign object being a flower, the same kind of flower has different meanings depending on the geographical region, the religion, the background, etc. In order to further improve the accuracy of the identified meaning, the execution body may acquire the geographical location information of the user, the religious belief information, the background information, etc., and integrate the above information to identify the meanings represented by various sub-sign object.

[0050] Next, reference is made to Fig. 4. Fig. 4 is a schematic diagram 400 showing an application scene of the method for identifying the meaning of a sign according to the present embodiment. In the application scene of Fig. 4, a user photographs a laundry label on a garment with a terminal device 401 to obtain a target image 402, and uploads the target image 402 to a server 403. The server 403 first determines whether or not the target image includes a sign object, and then, in response to determining that the target image includes an identification object, identifies the meaning represented by the sign object and returns the meaning to the terminal device.

[0051] In this embodiment, a method for identifying the meaning of a sign is provided, which automatically recognizes whether a sign object is included in a target image, and if the target image includes a sign object, the meaning represented by the sign object can be automatically identified. The sign efficiently represented by the user through the target image improves the efficiency of acquiring sign data and the efficiency of identifying the sign meaning.

[0052] In some optional implementations of this embodiment, the execution entity may execute an operation to present multiple types of sub-sign objects and multiple meanings that correspond one-to-one to the multiple types of sub-sign objects.

[0053] In this embodiment, the execution subject may present a plurality of types of sub-marker objects and a plurality of meanings that correspond one-to-one to the plurality of types of sub-marker objects, using a preset presentation method.

[0054] Here, the preset presentation method may be set according to the actual situation, for example, by presenting a plurality of sign objects and a plurality of meanings in one-to-one correspondence.

[0055] In this embodiment, based on the multiple types of sub-marker objects and multiple meanings presented, the user can quickly identify the meanings of various sub-marker objects, thereby improving the user's information acquisition efficiency.

[0056] In some optional implementations of this embodiment, the execution entity may execute the presentation process as follows.

[0057] In the first step, images of multiple sub-sign objects that have one-to-one correspondence with multiple types of sub-sign objects are recalled.

[0058] In one example, the execution entity or an electronic device communicatively connected to the execution entity is provided with a sign image set, the sign image set including reference images of various signs.

[0059] The execution entity can recall images of a plurality of sub-sign object that correspond one-to-one with the plurality of types of sub-sign objects from the sign image set.

[0060] In the second step, images of multiple sub-sign objects and their meanings are presented in the form of picture-text correspondence.

[0061] In this embodiment, the "figure" in the "figure-text correspondence format" refers to the image of the sub-sign object, and the "text" refers to its meaning. By adopting the figure-text correspondence format, the presentation effect of multiple sub-sign object images and multiple meanings is improved, which contributes to improving the user's information acquisition efficiency.

[0062] In some optional implementations of this embodiment, the execution entity may execute the first step as follows.

[0063] First, for each type of sub-signal object among a plurality of types, in response to determining that a preset sign image set contains an image of a sub-signal object corresponding to the type of sub-signal object, an image of the sub-signal object is recalled from the preset sign image set.

[0064] As an example, for each type of multiple types of sub-sign objects, the executing entity calculates the similarity between the feature data of the sub-sign object of that type and the feature data of the image of each sub-sign object in the preset sign image set, and if the maximum similarity is greater than the preset similarity threshold, identifies and recalls the image of the sub-sign object corresponding to the maximum similarity as an image of the sub-sign object corresponding to that type of sub-sign object, and if the maximum similarity is equal to or less than the preset similarity threshold, determines that the preset sign image set does not include an image of a sub-sign object corresponding to that type of sub-sign object.

[0065] Then, in response to determining that the preset sign image set does not include an image of a sub-sign object corresponding to the sub-sign object of that type, an image of a sub-sign object corresponding to the sub-sign object of that type is recalled from an image resource of the network.

[0066] The network image data is large in scale and has abundant resources. When it is determined that the image of the sub-signal object corresponding to the sub-signal object of the type is not included in the preset sign image set, the image of the sub-signal object corresponding to the sub-signal object of the type may be recalled from the image resources of the network by similarity-based recall.

[0067] In this embodiment, a method for recalling images of sub-sign objects is provided, and the accuracy of images of identified sub-sign objects is improved by combining a pre-defined set of sign images with the image resources of the network.

[0068] In some optional implementations of this embodiment, the execution entity may perform an operation of generating and presenting summary data for sign objects based on multiple types of sub-sign objects and multiple meanings that correspond one-to-one to the multiple types of sub-sign objects using a pre-trained artificial intelligence large-scale model, as follows:

[0069] Large-scale model (artificial intelligence large-scale model) specifically refers to a pre-trained large-scale model of artificial intelligence with large or super-large parameters. There are two meanings of artificial intelligence large-scale model: "pre-training" and "large-scale model". Combining both, a new artificial intelligence mode is born, that is, an artificial intelligence mode that can support various applications with only a small amount of data fine-tuning or no adjustment after the model is pre-trained with a large data set. The artificial intelligence large-scale model has excellent context understanding ability, language generation ability, learning ability, and transferability.

[0070] Taking the sign object as an example of a laundry mark, the executing entity can use the artificial intelligence large-scale model to generate comprehensive data for guiding a user on how to wash laundry based on a plurality of types of sub-sign objects and a plurality of meanings that correspond one-to-one to the plurality of sub-sign objects.

[0071] Taking the sign object as an example of a traffic sign, the executing entity can use the artificial intelligence large-scale model to generate summary data for guiding a user through the process of driving within an area corresponding to the sign object based on multiple types of sub-sign objects and multiple meanings that correspond one-to-one to the multiple types of sub-sign objects.

[0072] In this embodiment, the meanings of various sub-sign objects are presented to the user, and summary data is also presented, which not only resolves the user's main question of "What do the signs mean?", but also satisfies the user's secondary request of "presenting a prompt," further improving the user's information acquisition efficiency and experience.

[0073] In some optional implementations of this embodiment, the execution entity may execute a process for generating the summary data as follows.

[0074] First, the prompt of the artificial intelligence large-scale model is determined according to the scene represented by the sign object.

[0075] In this embodiment, a prompt corresponding to each scene may be set in advance. For example, for laundry label, the prompt information is "Please guide me on how to handle the laundry according to the laundry label xxx. Just summarize it in one sentence", and for flower sign, the prompt information is "Please tell me the number, name and corresponding flower language of xxx".

[0076] Once the scene that the sign object represents has been identified, a prompt for an artificial intelligence large scale model in that scene can be determined.

[0077] Then, the artificial intelligence large-scale model generates and presents summary data based on the prompt, multiple kinds of sub-sign objects and multiple meanings.

[0078] The summary data generated as an example of a prompt corresponding to the above laundry label is, for example, Garment care advice: wash as normal, hand wash in water below 30°C, do not bleach or tumble dry, iron at a temperature below 110°C, and dry cleaning is not recommended.

[0079] In this embodiment, by setting different prompts for different scenes, the large-scale model can generate and present summary data based on multiple types of sub-sign objects and multiple meanings, further improving the adaptability of the generated summary data to the scene.

[0080] In some optional implementations of this embodiment, the execution entity may perform the following operations.

[0081] First, in response to a determination that the target image does not include a marker object, other objects in the target image are identified.

[0082] Other objects are In the field of translation, texts to be translated, In the field of intelligent surveillance and security, people, vehicles or other objects to be monitored; Pedestrians, vehicles, obstacles, etc. in the field of autonomous driving, Lesions, tumors, and Vehicles, people, buildings, etc. in the drone photography field, In the agricultural and environmental observation field, we monitor crop growth conditions, areas where pests and diseases occur, etc. Including, but not limited to: Presents other objects and processed data for other objects in a two-column stream.

[0083] For the other objects, a processing may be performed on the target image in a corresponding processing manner to obtain processed data, and the other objects and the processed data for the other objects may be presented in a two-column stream.

[0084] In this embodiment, when the target image does not contain a sign object, other objects in the target image are identified and processed data for the other objects is presented, thereby improving the comprehensiveness of processing for the target image.

[0085] Reference is now made to Fig. 5, which shows a schematic flow 500 of a further embodiment of a method for identifying the meaning of a sign according to the present disclosure. The flow 500 includes the following steps:

[0086] In step 501, it is determined whether or not a marker object is included in the target image.

[0087] In step 502, in response to determining that a sign object is included in the target image, a category to which the sign object belongs is identified.

[0088] In step 503, a plurality of types of sub-sign objects belonging to a category that are included in the target image are identified.

[0089] In step 504, the meaning expressed by each of the multiple types of sub-sign objects is identified.

[0090] In step 505, for each type of sub-signal object of the plurality of types, in response to determining that the preset sign image set contains an image of a sub-signal object corresponding to that type of sub-signal object, an image of the sub-signal object is recalled from the preset sign image set.

[0091] In step 506, in response to determining that the preset sign image set does not include an image of a sub-sign object corresponding to the type of sub-sign object, an image of a sub-sign object corresponding to the type of sub-sign object is recalled from an image resource of the network.

[0092] In step 507, images of a plurality of sub-sign objects and a plurality of meanings are presented in a figure-text correspondence format.

[0093] In step 508, a prompt for the artificial intelligence large scale model is determined according to the scene represented by the sign object.

[0094] In step 509, the artificial intelligence large-scale model generates and presents summary data based on the prompt, the multiple sub-sign objects, and the multiple meanings.

[0095] As can be seen from this embodiment, compared with the embodiment corresponding to FIG. 2, the flow 500 of the method for identifying the meaning of a sign in this embodiment specifically describes the process of identifying the meaning of a sign and the process of generating summary data, which further improves the efficiency and experience of users' information acquisition.

[0096] Further referring to FIG. 6, as an embodiment of the method shown in each of the above figures, the present disclosure provides an embodiment of an apparatus for identifying the meaning of a sign, which embodiment of the apparatus corresponds to the embodiment of the method shown in FIG. 2, and which can be specifically applied to various electronic devices.

[0097] As shown in FIG. 6 , the apparatus 600 for identifying a sign meaning comprises a sign identification unit 601 configured to determine whether a target image includes a sign object, and a meaning identification unit 602 configured to identify a meaning represented by the sign object in response to determining that the target image includes the sign object.

[0098] In some optional implementations of this embodiment, the meaning identification unit 602 is further configured, in response to determining that the target image contains a sign object, to identify a category to which the sign object belongs, to identify multiple types of sub-sign object belonging to the category and contained in the target image, and to identify meanings represented by each of the multiple types of sub-sign object.

[0099] In some optional implementations of this embodiment, the meaning identification unit 602 is further configured to identify, for each type of sub-signal object of the plurality of types, a meaning represented by the sub-signal object of that type based on the number of sub-signal objects of that type in the target image.

[0100] In some optional implementations of this embodiment, the device may further comprise a presentation unit (not shown) configured to present a plurality of types of sub-sign objects and a plurality of meanings in one-to-one correspondence with the plurality of types of sub-sign objects.

[0101] In some optional implementations of this embodiment, the presentation unit is further configured to recall images of a plurality of sub-sign objects having one-to-one correspondence with the plurality of types of sub-sign objects, and present the images of the plurality of sub-sign objects and their meanings in a form of picture-text correspondence.

[0102] In some optional implementations of this embodiment, the presentation unit is further configured to, for each of the plurality of types of sub-signal objects, recall an image of the sub-signal object from the preset sign image set in response to determining that the preset sign image set includes an image of the sub-signal object corresponding to the type of sub-signal object, and to recall an image of the sub-signal object corresponding to the type of sub-signal object from an image resource of the network in response to determining that the preset sign image set does not include an image of the sub-signal object corresponding to the type of sub-signal object.

[0103] In some optional implementations of this embodiment, the above-mentioned apparatus further comprises a generation unit (not shown) configured to generate and present summary data for the sign object based on a plurality of types of sub-sign objects and a plurality of meanings having one-to-one correspondence with the plurality of types of sub-sign objects by a pre-trained artificial intelligence large-scale model.

[0104] In some optional implementations of this embodiment, the generation unit further determines a prompt for the artificial intelligence large-scale model based on the scene represented by the sign object, and generates and presents summary data by the artificial intelligence large-scale model based on the prompt, multiple types of sub-sign objects and multiple meanings.

[0105] In some optional implementations of this embodiment, the above apparatus further comprises an object identification unit (not shown) configured to identify other objects in the target image in response to determining that the target image does not include the sign object, and a data presentation unit (not shown) configured to present the other objects and processed data for the other objects in a two-column stream.

[0106] In this embodiment, a device for identifying the meaning of a sign is provided, which automatically recognizes whether a sign object is included in a target image, and if the target image includes a sign object, can automatically identify the meaning represented by the sign object. The sign efficiently represented by the user through the target image improves the efficiency of acquiring sign data and the efficiency of identifying the sign meaning.

[0107] According to an embodiment of the present disclosure, the present disclosure further provides an electronic device comprising at least one processor and a memory communicatively connected to the at least one processor, the memory storing instructions executable by the at least one processor, the instructions, when executed by the at least one processor, causing the at least one processor to perform a method for determining the meaning of a sign as described in any of the above embodiments.

[0108] According to an embodiment of the present disclosure, the present disclosure further provides a readable storage medium having stored thereon computer instructions for causing a computer to execute a method for determining the meaning of a sign as described in any of the above embodiments.

[0109] An embodiment of the present disclosure provides a computer program capable of implementing the method for determining the meaning of a sign according to any of the above embodiments when executed by a processor.

[0110] 7 shows a schematic block diagram of an exemplary electronic device 700 that can be used to implement embodiments of the present disclosure. The electronic device may represent various forms of numerical computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, mobile phones, smart phones, wearable devices, and other similar computing devices. It should be noted that the components shown, their connections, and their functions are merely exemplary and are not intended to limit the embodiments of the present disclosure described and / or claimed herein.

[0111] 7, the electronic device 700 comprises a computing unit 701 capable of performing various suitable operations and processes according to a computer program stored in a read only memory (ROM) 702 or loaded from a storage unit 708 into a random access memory (RAM) 703. The RAM 703 may further store various programs and data necessary for the operation of the device 700. The computing unit 701, the ROM 702 and the RAM 703 are connected to each other via a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0112] In the electronic device 700, several components including an input unit 706 such as a keyboard, a mouse, etc., an output unit 707 such as various types of displays, speakers, etc., a storage unit 708 such as a magnetic disk, an optical disk, etc., and a communication unit 709 such as a network plug-in, a modem, a wireless communication transceiver, etc. are connected to the I / O interface 705. The communication unit 709 enables the electronic device 700 to exchange information or data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0113] The computing unit 701 may be various general-purpose and / or special-purpose processing components having processing and computation capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that execute machine learning model algorithms, a digital signal processor (DSP), and any suitable processors, controllers, and microcontrollers. The computing unit 701 executes each method and process, such as the method for identifying the meaning of a sign, described above. For example, in some embodiments, the method for identifying the meaning of a sign may be realized as a computer software program tangibly included in a machine-readable medium, such as the storage unit 708. In some embodiments, some or all of the computer program may be loaded and / or installed in the electronic device 700 via the ROM 702 and / or the communication unit 709. When the computer program is loaded into the RAM 703 and executed by the computing unit 701, one or more steps of the method for identifying the meaning of a sign described above may be performed. Alternatively, in other embodiments, the computing unit 701 may be configured to perform the method for determining the meaning of the sign in any other suitable manner (eg, via firmware).

[0114] Various embodiments of the systems and techniques described herein may be realized in digital electronic circuitry systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. Each of these embodiments may be implemented in one or more computer programs that may be executed and / or interpreted in a programmable system that includes at least one programmable processor, which may be a special purpose or general purpose programmable processor, and may include receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0115] The program codes for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a general purpose computer, a special purpose computer, or other processor or controller for an apparatus for identifying the meaning of a sign, and when these program codes are executed by the processor or controller, the functions or operations defined in the flowcharts and / or block diagrams are performed. The program codes may be executed entirely on the device, partially on the device, partially on the device as a stand-alone software package and partially on a remote device, or entirely on a remote device or server.

[0116] In the context of this disclosure, a machine-readable medium may be a tangible medium, and may include or store a program for use by or in combination with an instruction-executing system, device, or apparatus. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or apparatus, or any suitable combination thereof. More specific examples of machine-readable storage media may include portable computer disks, hard disks, random access memories (RAMs), read-only memories (ROMs), erasable programmable read-only memories (EPROMs or flash memories), compact disk read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0117] To provide interaction with a user, the systems and techniques described herein may be implemented on a computer that includes a display device (e.g., a CRT (Cathode Ray Tube) or LCD (Liquid Crystal Display) monitor) for displaying information to a user, and a keyboard and pointing device (e.g., a mouse or trackball) through which a user can provide input to the computer. Other types of devices may also be used to interact with a user. For example, feedback provided to the user may be any form of sensing feedback, e.g., visual feedback, auditory feedback, or haptic feedback, and may receive input from the user in any form, including sound input, speech input, or haptic input.

[0118] The systems and techniques described herein may be implemented in a computing system including a back-end component (e.g., a data server), a computing system including a middleware component (e.g., an application server), a computing system including a front-end component (e.g., a user computer having a graphical user interface or a web browser) through which a user may interact with the embodiments of the systems and techniques described herein, or a computing system including any combination of such back-end, middleware, or front-end components. The components of the system may be connected by digital data communication through any form or medium, such as a communication network. Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0119] The computer system may include a client and a server. The client and the server are usually remote from each other and communicate with each other via a communication network. The relationship between the client and the server is generated by running computer programs having a client-server relationship with each other on the respective computers. The server may be a cloud server, also called a cloud computing server or cloud host, which is a host product in a cloud computing service system and is configured to solve the defects of high management difficulty and low business scalability in traditional physical hosts and virtual private server (VPS) services, and may be a server of a distributed system or a server combined with a blockchain.

[0120] According to the technical solution of an embodiment of the present disclosure, a method for identifying the meaning of a sign is provided, which can automatically recognize whether a sign object is included in a target image, and if the target image contains a sign object, can automatically identify the meaning represented by the sign object. The sign efficiently represented by the user through the target image improves the efficiency of acquiring sign data and the efficiency of identifying the meaning of the sign.

[0121] It should be understood that the above-mentioned various forms of flow can be used to rearrange, add, or remove steps. For example, each step described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution disclosed in this disclosure can be achieved. This specification is not limited here.

[0122] The above specific embodiments do not limit the scope of protection of the present disclosure. It should be understood that those skilled in the art can make various modifications, combinations, subcombinations, and replacements according to design requirements and other factors. Any modifications, equivalent replacements, improvements, etc. made without departing from the spirit and principle of the present disclosure should be included within the scope of protection of the present disclosure.

Claims

1. determining whether the target image includes a sign object; identifying a meaning expressed by the sign object in response to determining that the sign object is included in the target image; A method for identifying the meaning of signs, including:

2. The step of identifying a meaning expressed by the sign object in response to determining that the sign object is included in the target image includes: identifying a category to which the sign object belongs in response to determining that the sign object is included in the target image; Identifying a plurality of types of sub sign objects belonging to the category included in the target image; and identifying a meaning represented by each of the plurality of types of sub-sign objects.

3. The step of identifying the meaning represented by each of the plurality of types of sub-sign objects includes: The method according to claim 2 , further comprising a step of identifying, for each type of the plurality of types of sub-marker objects, a meaning expressed by the sub-marker object of the type depending on the number of sub-marker objects of the type in the target image.

4. The method of claim 2 , further comprising presenting a plurality of types of sub-sign objects and a plurality of meanings in one-to-one correspondence with the plurality of types of sub-sign objects.

5. The step of presenting the plurality of types of sub-sign objects and the plurality of meanings in one-to-one correspondence with the plurality of types of sub-sign objects includes: Recalling images of a plurality of sub-sign objects that correspond one-to-one to the plurality of types of sub-sign objects; The method of claim 4 , further comprising: presenting the images of the plurality of sub-sign objects and the plurality of meanings in a pictorial-text format.

6. The step of recalling images of a plurality of sub-sign objects that correspond one-to-one to the plurality of types of sub-sign objects includes: recalling an image of the sub-signal object from the preset sign image set in response to determining that a preset sign image set includes an image of a sub-signal object corresponding to the type of the sub-signal object for each of the plurality of types of sub-signal objects; The method according to claim 5, further comprising: recalling an image of a sub-sign object corresponding to the type of sub-sign object from an image resource of the network in response to determining that the preset sign image set does not include an image of a sub-sign object corresponding to the type of sub-sign object.

7. 3. The method of claim 2, further comprising: generating and presenting summary data for the sign object based on the plurality of sub-sign object types and a plurality of meanings that correspond one-to-one to the plurality of sub-sign object types using a pre-trained artificial intelligence large-scale model.

8. A step of generating and presenting summary data for the sign object based on the plurality of sub-sign objects and a plurality of meanings that correspond one-to-one to the plurality of sub-sign objects by a pre-trained artificial intelligence large-scale model, determining a prompt for the artificial intelligence large scale model according to the scene represented by the sign object; The method of claim 7, further comprising: generating and presenting the summary data based on the prompt, the plurality of sub-sign object types, and the plurality of meanings by the artificial intelligence large-scale model.

9. identifying another object in the target image in response to determining that the target image does not include the marker object; and presenting the other objects and the processed data for the other objects in a bi-column stream.

10. A device for identifying the meaning of a sign, comprising: a sign identification unit configured to determine whether a target image includes a sign object; a meaning identification unit configured to identify a meaning represented by the sign object in response to determining that the sign object is included in the target image; A device for determining the meaning of a sign comprising:

11. The meaning identification unit further comprises: The device according to claim 10, configured to, in response to determining that the target image contains the sign object, identify a category to which the sign object belongs, identify a plurality of types of sub-sign object belonging to the category that are included in the target image, and identify meanings expressed by each of the plurality of types of sub-sign object.

12. The meaning identification unit further comprises: The device according to claim 11 , configured to identify, for each type of the plurality of types of sub-marker objects, a meaning expressed by the sub-marker object of the type depending on the number of sub-marker objects of the type in the target image.

13. The apparatus of claim 11 , further comprising a presentation unit configured to present the plurality of types of sub-sign objects and a plurality of meanings having one-to-one correspondence with the plurality of types of sub-sign objects.

14. The presentation unit further comprises: The device according to claim 13, configured to recall images of a plurality of sub-sign objects having one-to-one correspondence with the plurality of types of sub-sign objects; and present the images of the plurality of sub-sign objects and the plurality of meanings in a pictorial-textual correspondence format.

15. The presentation unit further comprises: The device according to claim 14, configured to recall, for each type of the plurality of types of sub-signal objects, an image of the sub-signal object from the preset sign image set in response to determining that a sub-signal object image corresponding to the type of sub-signal object is included in the preset sign image set; and to recall, from an image resource of a network, an image of the sub-signal object corresponding to the type of sub-signal object in response to determining that the preset sign image set does not include an image of the sub-signal object corresponding to the type of sub-signal object.

16. The apparatus according to any one of claims 11 to 15, further comprising a generating unit configured to generate and present summary data for the identified object based on the plurality of types of sub-sign objects and a plurality of meanings having one-to-one correspondence with the plurality of types of sub-sign objects by a pre-trained artificial intelligence large-scale model.

17. The generating unit further comprises: The device of claim 16, further comprising: determining a prompt for the artificial intelligence large scale model based on the scene represented by the sign object; and configuring the artificial intelligence large scale model to generate and present the summary data based on the prompt, the plurality of types of sub-sign objects, and the plurality of meanings.

18. an object identification unit configured to identify other objects in the target image in response to determining that the target image does not include the marker object; The apparatus of claim 10 , further comprising: a data presentation unit configured to present the other objects and processed data for the other objects in a two-column stream.

19. At least one processor; a memory communicatively coupled to the at least one processor; An electronic device comprising:

10. An electronic device comprising: an input / output section for inputting a first input / output signal to the at least one processor; and a second input / output section for outputting a second input / output signal to the at least one processor;

20. A non-transitory computer-readable storage medium having computer instructions stored thereon, comprising: A non-transitory computer readable storage medium, the computer instructions causing a computer to perform the method of any one of claims 1 to 9.

21. A computer program product which, when executed by a processor, implements the method according to any one of claims 1 to 9.

Images