A focusing method, device, terminal device and storage medium for camera equipment

Abstract

The present application proposes a focusing method, device, terminal device and storage medium for a camera device, which belongs to the field of image processing technology. The method is applied to a terminal device, including: obtaining a first image and initial information, performing keyword detection on the initial information, and obtaining a target keyword corresponding to the initial information; performing target detection on the first image according to the target keyword to obtain target information corresponding to the first image; performing image segmentation on the first image according to the target information and the target keyword to obtain multiple second images; respectively calculating the information matching degree between the multiple second images and the initial information, and determining the target image corresponding to the initial information from the multiple second images according to the information matching degree; determining the position information corresponding to the target image from the first image, and determining the first target focus parameter corresponding to the camera device according to the position information; sending the first target focus parameter to the camera device, so that the camera device performs a focusing operation according to the first target focus parameter.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a focusing method, device, terminal device and storage medium for a camera device. Background Art

[0002] Taking USB cameras as an example, with the growing demand for remote office, online education, video conferencing, live broadcasting, etc., USB cameras play an important role in many fields. For example, USB cameras can be plug-and-play in video conferencing, which is convenient for users to conduct video conferencing through computers to facilitate remote office. In the field of education, USB cameras are widely used in remote teaching and online training. Teachers can interact with students in real time through USB cameras, providing students with a rich learning experience. However, USB cameras use a fixed window focus mode, so USB cameras cannot achieve free focus according to demand, which reduces the use effect of USB cameras. Summary of the invention

[0003] The present application aims to solve at least one of the technical problems existing in the prior art. To this end, the present application proposes a focusing method, device, terminal device and storage medium of a camera device, which can improve the free focus of the camera device and improve the use effect of the camera device.

[0004] In a first aspect, an embodiment of the present application provides a focusing method of a camera device, which is applied to a terminal device, and the method includes:

[0005] Obtaining a first image captured by the camera device that is communicatively connected to the terminal device and obtaining initial information required by a user to set a capture mode of the camera device through the terminal device;

[0006] Determine the information type corresponding to the initial information, and if the information type is a text type, perform keyword detection on the initial information to obtain a target keyword corresponding to the initial information;

[0007] Performing target detection on the first image according to the target keyword to obtain target information corresponding to the first image;

[0008] Performing image segmentation on the first image according to the target information and the target keyword to obtain a plurality of second images;

[0009] respectively calculating information matching degrees between the plurality of second images and the initial information, and determining a target image corresponding to the initial information from the plurality of second images according to the information matching degrees;

[0010] Determine position information corresponding to the target image from the first image, and determine a first target focus parameter corresponding to the camera device according to the position information;

[0011] The first target focus parameter is sent to the camera device, so that the camera device performs a focus operation according to the first target focus parameter.

[0012] In some optional embodiments, performing keyword detection on the initial information to obtain a target keyword corresponding to the initial information includes:

[0013] Performing entity recognition on the initial information using a named entity recognition model to obtain initial keywords corresponding to the initial information;

[0014] The initial keywords are subjected to entity uniform processing to obtain the target keywords corresponding to the initial information.

[0015] In some optional embodiments, the target information includes a target object and a target position corresponding to the target object, and the performing image segmentation on the first image according to the target information and the target keyword to obtain a plurality of second images includes:

[0016] Determine the target relationship between the target objects by using a relationship recognition model from the initial information in combination with the target keywords;

[0017] The first image is segmented according to the target relationship and the target position to obtain a plurality of the second images.

[0018] In some optional embodiments, respectively calculating the information matching degree between the plurality of second images and the initial information includes:

[0019] Determine a first image feature corresponding to the second image and a first text feature corresponding to the initial information by using an information feature representation layer of an information matching model;

[0020] Using the feature enhancement layer of the information matching model, performing image information enhancement on the first image feature to obtain a second image feature, and performing text information enhancement on the first text feature to obtain a second text feature;

[0021] The second image feature and the second text feature are semantically matched using the relevance matching layer of the information matching model to obtain the information matching degree between the second image and the initial information.

[0022] In some optional embodiments, performing semantic matching on the second image feature and the second text feature using the relevance matching layer of the information matching model to obtain the information matching degree between the second image and the initial information includes:

[0023] Calculating the correspondence between the local area corresponding to the second image feature and the word corresponding to the second text feature using the local matching unit of the correlation matching layer, and obtaining a first mutual matrix between the local area and the word;

[0024] Determine the attention image region vector corresponding to the second text feature using the text attention unit of the correlation matching layer;

[0025] Utilizing the first fusion unit of the correlation matching layer to calculate the first mutual matrix and the focus image region vector to determine a first local fusion vector;

[0026] Determine a text region vector corresponding to the second image feature using the image attention unit of the correlation matching layer;

[0027] Utilizing the second fusion unit of the correlation matching layer to calculate the first mutual matrix and the focus text region vector to determine a second local fusion vector;

[0028] Using the local fusion unit of the correlation matching layer to fuse the first local fusion vector and the second local fusion vector, to obtain a first matching degree corresponding to the second image feature and the second text feature;

[0029] Using the global text representation unit of the correlation matching layer to perform vector averaging on the vectors of the focus image region to obtain an average text vector, and performing linear projection on the average text vector to obtain a global text representation;

[0030] Using the global image representation unit of the correlation matching layer to perform vector averaging on the vector of the text region of interest to obtain an average image vector, and performing linear projection on the average image vector to obtain a global image representation;

[0031] Using the global fusion unit of the correlation matching layer to fuse the global text representation and the global image representation, to obtain a second matching degree corresponding to the second image feature and the second text feature;

[0032] The first matching degree and the second matching degree representation are fused using the overall fusion unit of the correlation matching layer to obtain the information matching degree corresponding to the second image feature and the second text feature.

[0033] In some optional embodiments, determining the first target focus parameter corresponding to the camera device according to the position information includes:

[0034] Determine a target window corresponding to the initial information according to the position information;

[0035] Determining, according to the target window, a focus center corresponding to when the camera device performs focusing;

[0036] The first target focus parameter corresponding to the camera device is determined according to the focus center and the target window.

[0037] In some optional embodiments, if the information type is an input instruction type, the method further includes:

[0038] Obtaining an image size corresponding to the first image;

[0039] Determine, according to the initial information, a trigger center corresponding to the user in the terminal device;

[0040] Determining the window size of the focus window when the camera device performs focusing;

[0041] Determine the initial focus parameter corresponding to the camera device according to the trigger center and the window size;

[0042] The initial focus parameter is adjusted according to the image size to obtain a second target focus parameter.

[0043] In a second aspect, an embodiment of the present application provides a focusing device of a camera device, comprising:

[0044] A data acquisition module is configured to obtain a first image captured by the camera device that is communicatively connected to the terminal device and to obtain initial information required by a user to set a capture mode of the camera device through the terminal device;

[0045] A data detection module is configured to determine the information type corresponding to the initial information, and if the information type is a text type, perform keyword detection on the initial information to obtain a target keyword corresponding to the initial information;

[0046] a target detection module, configured to perform target detection on the first image according to the target keyword to obtain target information corresponding to the first image;

[0047] an image segmentation module, configured to perform image segmentation on the first image according to the target information and the target keyword to obtain a plurality of second images;

[0048] an information matching module, configured to respectively calculate information matching degrees between the plurality of second images and the initial information, and determine a target image corresponding to the initial information from the plurality of second images according to the information matching degrees;

[0049] a parameter determination module, configured to determine position information corresponding to the target image from the first image, and determine a first target focus parameter corresponding to the camera device according to the position information;

[0050] The data sending module is configured to send the first target focus parameter to the camera device, so that the camera device performs a focus operation according to the first target focus parameter.

[0051] In some optional embodiments, the data detection module is further configured to:

[0052] An entity recognition module is configured to perform entity recognition on the initial information using a named entity recognition model to obtain initial keywords corresponding to the initial information;

[0053] The entity unification module is configured to perform entity unification processing on the initial keyword to obtain the target keyword corresponding to the initial information.

[0054] In some optional embodiments, the target information includes a target object and a target position corresponding to the target object, and the image segmentation module is further configured to:

[0055] a relationship extraction module configured to determine a target relationship between the target objects from the initial information in combination with the target keyword using a relationship recognition model;

[0056] The segmentation processing module is configured to perform image segmentation on the first image according to the target relationship and the target position to obtain a plurality of the second images.

[0057] In some optional embodiments, the information matching module is further configured to:

[0058] A feature representation module is configured to determine a first image feature corresponding to the second image and a first text feature corresponding to the initial information by using an information feature representation layer of an information matching model;

[0059] An information enhancement module is configured to use the feature enhancement layer of the information matching model to perform image information enhancement on the first image feature to obtain a second image feature and to perform text information enhancement on the first text feature to obtain a second text feature;

[0060] The semantic matching module is configured to perform semantic matching on the second image feature and the second text feature using the relevance matching layer of the information matching model to obtain the information matching degree between the second image and the initial information.

[0061] In some optional embodiments, the semantic matching module is further configured to:

[0062] a matrix acquisition module, configured to calculate the correspondence between the local area corresponding to the second image feature and the word corresponding to the second text feature by using the local matching unit of the correlation matching layer, and obtain a first mutual matrix between the local area and the word;

[0063] an image attention module, configured to determine an image attention region vector corresponding to the second text feature using the text attention unit of the correlation matching layer;

[0064] A first fusion module is configured to use the first fusion unit of the correlation matching layer to calculate the first mutual matrix and the focus image region vector to determine a first local fusion vector;

[0065] A focused text module, configured to determine a focused text region vector corresponding to the second image feature using the image attention unit of the correlation matching layer;

[0066] A second fusion module is configured to use the second fusion unit of the correlation matching layer to calculate the first mutual matrix and the focus text area vector to determine a second local fusion vector;

[0067] A first matching module is configured to fuse the first local fusion vector and the second local fusion vector by using the local fusion unit of the correlation matching layer to obtain a first matching degree corresponding to the second image feature and the second text feature;

[0068] A text representation module is configured to perform vector averaging on the focus image region vector using the global text representation unit of the correlation matching layer to obtain an average text vector, and perform linear projection on the average text vector to obtain a global text representation;

[0069] An image representation module is configured to perform vector averaging on the focus text region vector using the global image representation unit of the correlation matching layer to obtain an average image vector, and perform linear projection on the average image vector to obtain a global image representation;

[0070] A second matching module is configured to fuse the global text representation and the global image representation using the global fusion unit of the correlation matching layer to obtain a second matching degree corresponding to the second image feature and the second text feature;

[0071] The target matching module is configured to fuse the first matching degree and the second matching degree representation using the overall fusion unit of the correlation matching layer to obtain the information matching degree corresponding to the second image feature and the second text feature.

[0072] In some optional embodiments, the parameter determination module is configured to:

[0073] a window determination module, configured to determine a target window corresponding to the initial information according to the position information;

[0074] A first center determination module is configured to determine a focus center corresponding to when the camera device focuses according to the target window;

[0075] The first parameter determination module is configured to determine the first target focus parameter corresponding to the camera device according to the focus center and the target window.

[0076] In some optional embodiments, the focusing device of the camera device is further configured as follows:

[0077] an image determination module, configured to obtain an image size corresponding to the first image;

[0078] A second center determination module is configured to determine a trigger center corresponding to the user in the terminal device according to the initial information;

[0079] A size determination module, configured to determine the window size of the focus window when the camera device performs focusing;

[0080] A second parameter determination module is configured to determine an initial focus parameter corresponding to the camera device according to the trigger center and the window size;

[0081] The parameter adjustment module is configured to adjust the initial focus parameter according to the image size to obtain a second target focus parameter.

[0082] In a third aspect, an embodiment of the present application provides a terminal device, comprising at least one control processor and a memory for communicating with the at least one control processor; the memory stores instructions that can be executed by the at least one control processor, and the instructions are executed by the at least one control processor so that the at least one control processor can execute the focusing method of the camera device as described in the first aspect above.

[0083] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium storing computer-executable instructions, wherein the computer-executable instructions are used to execute the focusing method of the camera device as described in the first aspect above.

[0084] According to the focusing method of the camera device of the embodiment of the present application, there are at least the following beneficial effects: the method includes obtaining a first image captured by a camera device connected to a terminal device for communication and obtaining initial information required by a user to set the camera device capture mode through the terminal device; determining the information type corresponding to the initial information, if the information type is a text type, performing keyword detection on the initial information to obtain the target keyword corresponding to the initial information; performing target detection on the first image according to the target keyword to obtain the target information corresponding to the first image; performing image segmentation on the first image according to the target information and the target keyword to obtain multiple second images; calculating the information matching degree between the multiple second images and the initial information respectively, and determining the target image corresponding to the initial information from the multiple second images according to the information matching degree; determining the position information corresponding to the target image from the first image, and determining the first target focus parameter corresponding to the camera device according to the position information; sending the first target focus parameter to the camera device, so that the camera device performs a focusing operation according to the first target focus parameter. The method allows the user to perform keyword detection on the initial information according to the initial information required by the terminal device when capturing the camera device, which helps to accurately identify the user's needs or intentions, so as to perform focusing processing according to the user's needs later. By performing target detection on the first image, the target object in the image can be identified, which is helpful for subsequent image segmentation and focusing operations. According to the target information and target keywords, the first image can be segmented into multiple second images, and then by calculating the information matching degree between the multiple second images and the initial information, it can be determined which second image best meets the user's needs, thereby determining the target image. Finally, based on the position information of the target image, the first target focus parameter of the camera device is determined to ensure that the camera device can accurately focus on the target object according to the user's needs, achieve the purpose of user free focus, and thus achieve the user's needs or intentions. The problem that the camera device cannot achieve free focus in the related technology and reduces the user's use effect is solved, and the free focus of the camera device is achieved, the effect of the user using the camera device is improved, and the user satisfaction is further improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0085] Figure 1 A schematic diagram of a flow chart of a focusing method of a camera device provided by an embodiment of the present invention;

[0086] Figure 2 A schematic diagram of the structure of an information matching model provided by an embodiment of the present invention;

[0087] Figure 3 A schematic diagram of the structure of another information matching model provided by an embodiment of the present invention;

[0088] Figure 4 A schematic diagram of a scene of a focusing method of a camera device provided by an embodiment of the present invention;

[0089] Figure 5 A schematic diagram of the contents of the UVC private protocol provided in an embodiment of the present invention;

[0090] Figure 6 A schematic diagram of the module structure of a focusing device for a camera device provided by an embodiment of the present invention;

[0091] Figure 7 A schematic block diagram of the structure of a terminal device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0092] The embodiments of the present application are described in detail below, and examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present application, and cannot be understood as limiting the present application.

[0093] In the description of the present application, it should be understood that descriptions involving orientation, such as up, down, front, back, left, right, etc., indicating orientations or positional relationships, are based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as a limitation on the present application.

[0094] In the description of this application, "several" means one or more, "more" means more than two, "greater than", "less than", "exceed", etc. are understood to exclude the number itself, and "above", "below", "within", etc. are understood to include the number itself. If there is a description of "first" or "second", it is only used for the purpose of distinguishing technical features, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0095] In the description of this application, unless otherwise clearly defined, terms such as setting, installing, connecting, etc. should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meanings of the above terms in this application based on the specific content of the technical solution.

[0096] In order to make the description of the following embodiments clear and concise, a brief introduction to the related technology is first given:

[0097] When the camera device is a mobile phone, when the mobile phone implements the multi-point AF (Auto Focus, free focus) focusing function, it usually adopts the algorithms and mechanisms built into the mobile phone operating system and camera application. These algorithms and mechanisms may be based on different technical principles and implementation methods, and there are significant differences from the camera system implemented through private protocols or UVC protocols. When implementing multi-point AF focusing, mobile phones may be limited by factors such as hardware performance, screen size and resolution. This may cause a gap between mobile phones and professional camera systems in terms of focusing speed, accuracy and flexibility. Mobile phone camera applications usually provide preset focus modes and options, and users cannot freely set the AF window like through private protocols. This limits the focusing ability of mobile phones in complex scenes and the space for personalized adjustment, making free focus impossible.

[0098] When the camera device is a USB camera, the existing focus technology of the USB camera can only preset the AF window inside the ISP (Image Singal Process). That is to say, when the USB camera leaves the factory, the AF window is fixed. In actual use, the user cannot change the corresponding AF window according to actual needs. On the basis of affecting the user experience, it further limits the application scenarios of the USB camera. The specific manifestations are as follows: (1) Focus area limitation: Fixed window focus means that the focus area is preset and cannot be adjusted. This results in the user being unable to freely select the focus area according to shooting needs, especially when the shooting scene is complex or specific details need to be accurately focused, the fixed window focus may not meet the requirements; (2) Focus accuracy problem: Due to the fixed focus area, the fixed window focus may not be able to achieve accurate focus in all situations. In particular, when the subject is not within the preset focus area or the scene light is poor, inaccurate focus or blur may occur, affecting the quality of the photo; (3) Lack of flexibility: Fixed window focus lacks flexibility and cannot adapt to changes in different shooting scenes and objects. For example, when shooting moving objects or scenes that require continuous focus, fixed window focus may not provide sufficient tracking and stability, resulting in reduced photo or video quality; (4) Inconvenient operation: For users, fixed window focus may mean more manual adjustments and post-processing to compensate for the lack of focus, which increases the complexity and time cost of shooting and reduces the user experience.

[0099] In summary, although the mobile phone mode focus technology or fixed window focus technology may be applicable in certain specific scenarios, its limitations and inflexibility make it impossible for the camera device to focus freely, reducing the user's usage effect and experience results.

[0100] According to the above introduction to the related art, it can be seen that in the prior art, there is a problem that the camera device cannot achieve free focus, which reduces the user experience. In view of the problem that the camera device cannot achieve free focus in the related art, which reduces the user experience, the embodiment of the present application provides a focusing method, device, terminal device and storage medium for a camera device.

[0101] In the present application, a focusing method, an apparatus, a terminal device and a storage medium of a camera device are provided, which are described in detail one by one in the following embodiments.

[0102] Reference Figure 1 , Figure 1 A flowchart of a focusing method of a camera device provided in an embodiment of the present application is applied to a terminal device. The focusing method of the camera device includes but is not limited to the following steps:

[0103] Step S101, obtaining a first image captured by the camera device that is communicatively connected to the terminal device and obtaining initial information required by a user to set a capture mode of the camera device through the terminal device.

[0104] Exemplarily, the camera device is a USB camera, which is inserted into the terminal device through a USB interface, and then after the user adjusts the shooting angle of the USB camera, the USB camera captures the first image at the current shooting angle, and sends the first image to the terminal device through a USB transmission protocol, so that the terminal device obtains the first image captured by the camera device. The purpose of obtaining the first image captured by the camera device that is communicatively connected to the terminal device in the present application is to more clearly understand the real scene that can be captured by the current camera device, and to provide good support for subsequently obtaining the first target focus parameters of the camera device that are more accurate and close to the real scene.

[0105] For example, when the camera device is a USB camera, the user cannot directly operate the USB camera to achieve the purpose of focusing. Therefore, the present application can obtain the initial information required for the camera device's acquisition method with the help of the terminal device into which the USB camera is inserted, thereby realizing free interaction between the USB camera and the terminal device, providing a basis for subsequent users to achieve free focus according to their own needs.

[0106] It should be noted that the first image can be an image of the current moment captured by the camera device, or it can be any image selected by the user from a historical moment captured by the camera device, etc. This application does not make specific restrictions, and users can set it according to actual needs.

[0107] It should be noted that the initial information can be text information, gesture information, voice information, etc. This application does not make specific restrictions, and users can set it according to actual needs.

[0108] Step S102, determining the information type corresponding to the initial information, and if the information type is a text type, performing keyword detection on the initial information to obtain target keywords corresponding to the initial information.

[0109] Exemplarily, the information type corresponding to the initial information is obtained according to the storage format of the initial information. For example, when the initial information is voice information, the storage format is wave or mp3, when the initial information is text type, the storage format is txt or string, and when the initial information is gesture information, the storage format is img or video.

[0110] Exemplarily, if the information type is text type, an appropriate word segmentation tool is used on the initial information to divide the initial information into words or phrases, and then stop words in the initial information are removed. Stop words are words that have no practical meaning in the initial information. Finally, TF-IDF, TextRank, or deep learning models (such as BERT) and other technologies are used to extract keywords, so as to select the most relevant or most frequently occurring keywords as target keywords according to needs.

[0111] In some optional embodiments, the performing keyword detection on the initial information to obtain the target keyword corresponding to the initial information includes: performing entity recognition on the initial information using a named entity recognition model to obtain the initial keyword corresponding to the initial information; performing entity unified processing on the initial keyword to obtain the target keyword corresponding to the initial information.

[0112] Exemplarily, a named entity recognition (NER) model is used to perform entity recognition on the initial information. The NER model can recognize entities with specific meanings in the text. In this application, the entity types in the initial information can be divided into character descriptions, scene descriptions, environment descriptions, vehicle descriptions, etc., and then a pre-trained deep learning model such as BERT, RoBERTa, etc. is selected. Based on this deep learning model, model incremental training is first performed to obtain a NER model suitable for entity type recognition in this application, so that the initial information is recognized according to the NER model to obtain the initial keywords corresponding to the initial information.

[0113] For example, entity recognition may generate entity names in various forms. In order to facilitate the subsequent acquisition of more accurate focus parameters, the entities can be unified to provide support for the subsequent provision of more accurate focus parameters.

[0114] For example, knowledge bases or rules can be used to unify entities of different forms into a standard form. For example, different character descriptions or scene descriptions can be unified into a standard name. Among them, entity unification can improve the accuracy of subsequent processing and avoid multiple forms of the same entity being regarded as different keywords, which affects the subsequent image and text fusion.

[0115] For example, after unified entity processing, target keywords corresponding to the initial information can be obtained. These target keywords can be used as representatives of important entities in the text for subsequent analysis or processing.

[0116] Specifically, named entity recognition can identify important entities in the initial information, thereby improving the accuracy of the target keywords. Entity unification processing can unify different forms of initial keywords into the same standard form, avoiding repeated keywords and the influence of different forms of keywords on the subsequent image and text fusion results. Finally, entity recognition and unification make the target keywords more semantically informative, which helps to understand the meaning and context of the initial information. This provides support for providing more accurate focus parameters in the future.

[0117] Step S103: performing target detection on the first image according to the target keyword to obtain target information corresponding to the first image.

[0118] Exemplarily, the first image is processed using a target detection model to identify a target object in the first image. The target detection model can be a model based on traditional computer vision methods, such as Haar cascade detector, HOG feature + SVM classifier, etc., or a model based on deep learning, such as Faster R-CNN, YOLO, SSD, etc. Among them, the type of target object recognition performed by the target detection model corresponds to the entity type for entity recognition in the naming recognition model.

[0119] Exemplarily, the first image is processed using a target detection model to obtain a target detection result of the target detection model, and information of the target object detected in the first image is extracted. This information may include the location of the target (such as bounding box coordinates), category label, confidence, etc., and then by comparing the category label and target keyword in the target detection result, the target object related to the target keyword is found, so that the information of the target object in the target detection result is determined as the target information corresponding to the first image. Therefore, by matching the target detection result with the target keyword, it can be ensured that the obtained target information is associated with the target keyword, and the initial information is further associated with the first image.

[0120] Step S104: segment the first image according to the target information and the target keyword to obtain a plurality of second images.

[0121] Exemplarily, the target information is combined according to the target keywords, and the first image is segmented according to the combination result to obtain a plurality of second images, wherein there may be an image overlap region between any two second images.

[0122] Exemplarily, when combining target information according to target keywords, it is possible to consider whether there is connectivity between the target information, that is, whether the target information is connected, to avoid the second image obtained after data combination being the same as the first image, thereby failing to achieve the purpose of free focusing according to the initial information.

[0123] In some optional embodiments, the target information includes a target object and a target position corresponding to the target object, and performing image segmentation on the first image according to the target information and the target keyword to obtain multiple second images includes: using a relationship recognition model to determine the target relationship between the target objects from the initial information in combination with the target keyword; performing image segmentation on the first image according to the target relationship and the target position to obtain multiple second images.

[0124] Exemplarily, the target information includes a target object and a target position corresponding to the target object, wherein the target object is a target object corresponding to a target keyword, for example, if the target keyword is a person, then the target object is the person in the first image, and the target information is the person and the target position of the person in the first image.

[0125] Exemplarily, the relationship between the target keywords in the initial information is identified using the relationship identification model to obtain the association relationship between the target keywords. Since the target keywords correspond to the target objects, the association relationship between the target keywords can be determined as the target relationship between the target objects corresponding to the target keywords.

[0126] For example, the relationship recognition model is a model based on deep learning, such as BERT, GPT, etc., or you can design your own model according to task requirements, so as to determine the target relationship between target objects through the relationship recognition model, such as adjacent relationship, upper and lower relationship, surrounding relationship, etc.

[0127] Exemplarily, the target objects are combined according to the target relationship and the target positions corresponding to the target objects to obtain the combination results corresponding to the target objects, and then the image is segmented according to the combination results to obtain multiple second images. The target position here refers to the position information of the target object in the first image.

[0128] For example, if the target relationship is an adjacent relationship, the combination method between the target objects is determined according to the target positions. When target position 1 corresponding to target object 1 and target position 2 corresponding to target object 2 are not adjacent, target object 1 and target object 2 cannot be combined; when target position 2 corresponding to target object 2 and target position 3 corresponding to target object 3 are adjacent, target object 2 and target object 3 can be combined.

[0129] Exemplarily, the target objects are combined according to the target relationship and the target position to obtain a combination result, and then the first image is segmented according to the combination result to obtain a plurality of second images.

[0130] Specifically, by extracting the target relationship between target objects from the initial information in combination with the target keywords, the initial information can be understood more comprehensively, thereby enabling more accurate subsequent processing. By segmenting the first image according to the target relationship and target position, the image area related to the target relationship can be obtained, thereby achieving refined processing of the first image, and the semantic information in the initial information can be more effectively utilized, thereby improving the efficiency and accuracy of the initial information utilization. This provides good support for the subsequent identification of focus parameters.

[0131] Step S105 , respectively calculating information matching degrees between the plurality of second images and the initial information, and determining a target image corresponding to the initial information from the plurality of second images according to the information matching degrees.

[0132] Exemplarily, a metric or similarity algorithm is defined to calculate the information matching degree between the second image and the initial information, which can be a similarity calculation based on image features, or a similarity calculation based on text information, or a comprehensive matching degree calculation combining the two.

[0133] For example, the second image is converted into a corresponding text description, thereby calculating the information matching degree between the converted text description and the initial information, or the image information corresponding to the initial information is generated, thereby calculating the information matching degree between the image information and the second image. Through the information matching degree calculation, the matching degree between each second image and the initial information can be quantitatively evaluated, thereby accurately determining the target image corresponding to the initial information.

[0134] Exemplarily, according to the information matching degree, the second image with the highest matching degree is selected as the target image corresponding to the initial information. For example, a threshold is set, and the second image with a matching degree higher than the threshold is selected as the candidate target image. If there are multiple second images with matching degrees higher than the threshold, the image with the highest matching degree can be selected as the final target image.

[0135] In some optional embodiments, respectively calculating the information matching degree between the plurality of second images and the initial information includes: using the information feature representation layer of the information matching model to determine the first image feature corresponding to the second image and the first text feature corresponding to the initial information; using the feature enhancement layer of the information matching model to perform image information enhancement on the first image feature to obtain the second image feature and to perform text information enhancement on the first text feature to obtain the second text feature; using the relevance matching layer of the information matching model to perform semantic matching on the second image feature and the second text feature to obtain the information matching degree between the second image and the initial information.

[0136] For example, Figure 2 As shown, the information matching model includes an information feature representation layer, a feature enhancement layer, and a relevance matching layer. First, the information feature representation layer of the information matching model is used to extract the image features corresponding to the second image and the text features corresponding to the initial information to obtain the first image features corresponding to the second image and the first text features corresponding to the initial information. The feature enhancement layer of the information matching model is then used to enhance the image information of the first image features to obtain the second image features and to enhance the text information of the first text features to obtain the second text features. Finally, the relevance matching layer of the information matching model is used to perform semantic matching on the second image features and the second text features, thereby obtaining the information matching degree between the second image and the initial information.

[0137] Exemplarily, the feature enhancement layer of the information matching model is used to enhance the first image feature to obtain the second image feature. Image information enhancement aims to improve the image quality and enhance the image features through various image processing techniques to improve the effect of subsequent processing. For example, the weight of the target object in the second image is strengthened to obtain the second image feature.

[0138] Exemplarily, the feature enhancement layer of the information matching model is used to enhance the first text feature to obtain the second text feature. The purpose of text information enhancement is to improve the quality of the text and enhance the features of the text through various natural language processing technologies to improve the effect of subsequent processing. For example, the weight of the target keyword in the initial information is strengthened to obtain the second text feature. Through these steps of image information enhancement and text information enhancement, the second image feature and the second text feature can be made more suitable for subsequent processing and analysis, thereby improving the quality and availability of the information.

[0139] Exemplarily, the relevance matching layer of the information matching model is used to calculate the cosine distance between the second image feature and the second text feature to obtain the information matching degree between the second image and the initial information.

[0140] It should be noted that, in order to obtain multiple second images in this application, it is necessary to semantically match each second image with the initial information, so as to obtain the information matching degree between each second image and the initial information. The number of information matching degrees finally obtained is the same as the number of second images.

[0141] In addition, it should be noted that before using the information matching model, technical details such as model training, parameter tuning and result evaluation need to be considered to ensure the smooth implementation and effectiveness verification of the entire process, which will not be repeated in this application.

[0142] Specifically, the information feature representation layer of the information matching model can effectively extract the information features of the second image and the initial information. Through the processing of the feature enhancement layer, the second image and the initial information can be facilitated to perform semantic matching, thereby improving the accuracy and reliability of the matching. Finally, the semantic matching using the correlation matching layer can quantitatively evaluate the degree of semantic association between the second image and the initial information, providing a reliable basis for subsequent decision-making.

[0143] In some optional embodiments, the use of the correlation matching layer of the information matching model to perform semantic matching on the second image feature and the second text feature to obtain the information matching degree between the second image and the initial information includes: using the local matching unit of the correlation matching layer to calculate the correspondence between the local area corresponding to the second image feature and the word corresponding to the second text feature to obtain a first mutual matrix between the local area and the word; using the text attention unit of the correlation matching layer to determine the image area vector corresponding to the second text feature; using the first fusion unit of the correlation matching layer to calculate the first mutual matrix and the image area vector to determine the first local fusion vector; using the image attention unit of the correlation matching layer to determine the text area vector corresponding to the second image feature; using the second fusion unit of the correlation matching layer to calculate the first mutual matrix and the text area vector to determine the second local fusion vector; using the correlation The local fusion unit of the matching layer fuses the first local fusion vector and the second local fusion vector to obtain the first matching degree corresponding to the second image feature and the second text feature; the global text representation unit of the correlation matching layer is used to perform vector averaging on the focus image area vector to obtain an average text vector, and the average text vector is linearly projected to obtain a global text representation; the global image representation unit of the correlation matching layer is used to perform vector averaging on the focus text area vector to obtain an average image vector, and the average image vector is linearly projected to obtain a global image representation; the global fusion unit of the correlation matching layer is used to fuse the global text representation and the global image representation to obtain the second matching degree corresponding to the second image feature and the second text feature; the overall fusion unit of the correlation matching layer is used to fuse the first matching degree and the second matching degree representation to obtain the information matching degree corresponding to the second image feature and the second text feature.

[0144] For example, Figure 3 As shown, the correlation matching layer includes a local matching unit, a text attention unit, a first fusion unit, an image attention unit, a second fusion unit, a local fusion unit, a global text representation unit, a global image representation unit, a global fusion unit, and an overall fusion unit. The local matching unit of the correlation matching layer is used to calculate the cosine similarity between any local area in the second image feature and any word in the second text feature, so that the value of the cosine similarity is determined as the matrix value in the corresponding first mutual matrix between the local area corresponding to the second image feature and the word corresponding to the second text feature. By constructing the first mutual matrix, the fine-grained interaction between the local area corresponding to the second image feature and the word corresponding to the second text feature can be further inferred.

[0145] Exemplarily, in order to focus on the words in each local area, the text attention unit of the correlation matching layer is also used to calculate the information of the focus words in each local area, for example, by weighted combination of each second text feature to determine the focus image area vector corresponding to the second text feature.

[0146] Exemplarily, the first local fusion vector is calculated by multiplying the first mutual matrix and the focus image region vector using the first fusion unit of the correlation matching layer.

[0147] Exemplarily, in order to focus on the local area corresponding to each word, the image attention unit of the correlation matching layer can also be used to perform weighted combination on the second image features, so as to determine the focus text area vector corresponding to the second image features.

[0148] Exemplarily, in order to determine the importance of the context in the initial information to each local area, the cosine similarity between any word in the second text feature and any local area in the second image feature is calculated, so that the value of the cosine similarity is determined as the matrix value in the corresponding second mutual matrix between the word corresponding to the second text feature and the local area corresponding to the second image feature. By constructing the second mutual matrix, the fine-grained interaction between the local area corresponding to the second image feature and the word corresponding to the second text feature can be further inferred. In addition, the second mutual matrix is ​​the transpose of the first matrix. Therefore, when the second local fusion vector is calculated using the second fusion unit of the correlation matching layer, it is necessary to multiply the transpose of the first mutual matrix and the focus text area vector to determine the second local fusion vector.

[0149] Exemplarily, the local fusion unit of the correlation matching layer is used to perform weighted fusion on the first local fusion vector and the second local fusion vector to obtain a first matching degree corresponding to the second image feature and the second text feature.

[0150] Exemplarily, in order to explore a deeper level of image-text interaction, it is also necessary to use the global text representation unit of the correlation matching layer to add and average all vectors in the vector of the focused image region to obtain an average text vector, and linearly project the average text vector to obtain a global text representation. And the global image representation unit of the correlation matching layer is used to add and average all vectors in the vector of the focused text region to obtain an average image vector, and linearly project the average image vector to obtain a global image representation. Among them, the first linear parameter required for linear projection of the average text vector and the second linear parameter required for linear projection of the average image vector are obtained when the information matching model is trained.

[0151] Exemplarily, a global fusion unit of a correlation matching layer is used to perform weighted fusion on the global text representation and the global image representation to obtain a second matching degree corresponding to the second image feature and the second text feature.

[0152] Exemplarily, the overall fusion unit of the correlation matching layer is finally used to fuse the local and overall results of the first matching degree and the second matching degree, so as to obtain the information matching degree corresponding to the second image feature and the second text feature.

[0153] Specifically, using the local matching unit to calculate the correspondence between the local region and the word between the second image feature and the second text feature can help the model capture the local semantic information between the image and the text more accurately, thereby improving the matching accuracy. Using the text attention unit and the image attention unit to determine the attention image region vector corresponding to the second text feature and the attention text region vector corresponding to the second image feature can help the model focus on the area that has the greatest impact on the matching degree, thereby improving the efficiency and accuracy of the model. By fusing the information of the local region and the text word through the first fusion unit and the second fusion unit, the degree of association between the local information can be comprehensively considered, thereby better representing the matching between the image and the text. Using the global text representation unit and the global image representation unit to perform vector averaging and linear projection on the attention region vector to obtain the global text representation and the global image representation can help the model better understand the overall semantic information, thereby improving the accuracy of the matching degree. By fusing the global text representation and the global image representation through the global fusion unit, the global semantic information of the text and the image can be combined, further improving the accuracy and robustness of the matching degree. By fusing the first matching degree and the second matching degree through the overall fusion unit, the matching degree of local and global information can be comprehensively considered, thereby obtaining a more comprehensive and accurate information matching degree.

[0154] Step S106: determining position information corresponding to the target image from the first image, and determining a first target focusing parameter corresponding to the camera device according to the position information.

[0155] Exemplarily, the position information corresponding to the target image is obtained from the first image, that is, the position information of the image area represented by the initial information is obtained from the first image, so that the position information is determined as the area clicked by the user that needs to be focused, and the position information is converted into the first target focus parameter required for the camera device to focus.

[0156] For example, the first target focus parameter includes the horizontal starting coordinate, vertical starting coordinate, horizontal length and vertical length of the free focus window. Then, the horizontal coordinate of the upper left corner in the position information is converted to the horizontal starting coordinate of the free focus window, the vertical coordinate of the upper left corner is converted to the vertical starting coordinate, the horizontal width in the position information is converted to the horizontal length of the free focus window, and the vertical width in the position information is converted to the vertical length of the free focus window.

[0157] In some optional embodiments, determining the first target focus parameter corresponding to the camera device based on the position information includes: determining a target window corresponding to the initial information based on the position information; determining a focus center corresponding to the camera device when focusing based on the target window; and determining the first target focus parameter corresponding to the camera device based on the focus center and the target window.

[0158] Exemplarily, the position information includes the starting position corresponding to the target image and the size of the target image, so that the target window corresponding to the initial information in the first image can be determined according to the position information. This helps to accurately locate the target area of ​​interest to the user in the image, and then connect the diagonal positions in the target window, so as to determine the intersection between the two diagonal lines obtained as the focus center corresponding to the camera when focusing, thereby ensuring accurate focus on the target area during the camera process, avoiding the problem of image blur or distortion due to inaccurate focal length, and improving the quality and clarity of the image.

[0159] Exemplarily, the maximum window length and maximum window width corresponding to the camera device when focusing are obtained, the minimum value between the current window length and the maximum window length of the target window is determined as the target window length, and the minimum value between the current window width and the maximum window width of the target window is determined as the target window width. Thus, the first target focus parameter corresponding to the camera device is determined according to the focus center, the target window length, and the target window width.

[0160] Specifically, the position information of the target image can ensure that the camera device can accurately focus on the area corresponding to the initial information, thereby improving the clarity and quality of the image captured by the camera device and better meeting the actual needs of users.

[0161] Step S107: sending the first target focus parameter to the camera device, so that the camera device performs a focus operation according to the first target focus parameter.

[0162] Exemplarily, a communication channel is established between the terminal device and the camera device, and various communication methods, such as Wi-Fi, Bluetooth, wired connection, etc., can be used to ensure that data can be transmitted in both directions. When the camera device is a USB camera, the data transmission between the terminal device and the camera device can be realized through the UVC private protocol (USB Video Class, which is a USB device class video standard). Thus, the first target focus parameter is converted into the data format required by the UVC private protocol, and the first target focus parameter can be encoded into a string or binary data in a pre-defined format so that the camera device can correctly parse and process it.

[0163] Exemplarily, the encoded first target focus parameter is sent to the camera device through the established communication channel to ensure that the data can be quickly and stably transmitted between the terminal device and the camera device. After receiving the encoding result, the camera device parses and processes it to ensure that the first target focus parameter can be correctly extracted and the focus operation is performed according to the first target focus parameter. The camera device performs the focus operation according to the received first target focus parameter, adjusts the focal length and focus, and ensures that the target is clearly visible.

[0164] In addition, after the camera device performs a focusing operation according to the first target focus parameters, it can also capture a third image based on the camera device under the current focusing operation, and feed the third image back to the terminal device, so that the terminal device displays the third image, allowing the user to determine whether the initial information set by the user is met based on the display result.

[0165] In some optional embodiments, if the information type is an input instruction type, the method also includes: obtaining the image size corresponding to the first image; determining the trigger center corresponding to the user in the terminal device based on the initial information; determining the window size of the focus window when the camera device focuses; determining the initial focus parameters corresponding to the camera device based on the trigger center and the window size; adjusting the initial focus parameters according to the image size to obtain second target focus parameters.

[0166] Exemplarily, if the information type is an input instruction type, that is, the initial information is that the user uses the terminal device to select the focus center in the first image by touching or clicking, so that the position touched or clicked by the user is converted into the corresponding trigger center. The window size of the focus window when the camera device focuses is obtained from the database, that is, the window width and the window length. Therefore, the initial focus parameters corresponding to the camera device are determined according to the window size and the trigger center.

[0167] Exemplarily, the image size corresponding to the first image is obtained, and when the window information is generated by the trigger center and the window size, the trigger center is determined as the window center, and then the corresponding target window is generated according to the window size and the window center. When the target window exceeds the image size corresponding to the first image, that is, part of the target window is already distributed outside the first image, it is necessary to adjust the initial focus parameters to obtain the second target focus parameters.

[0168] Exemplarily, the image size corresponding to the first image is obtained through a corresponding image processing library or API. Then, based on the initial information, the trigger center corresponding to the user in the terminal device is determined, and then the area or target of interest to the user is obtained, providing guidance for subsequent focusing operations. The window size of the focus window when the camera device focuses is determined according to the performance of the camera device. This can adjust the focal length range according to actual needs to ensure that the focusing operation can cover the area of ​​interest to the user.

[0169] Exemplarily, the initial focus parameter corresponding to the camera device is determined according to the trigger center and the window size. This helps to limit the focus range to the area of ​​interest to the user and improve the efficiency and accuracy of the focus operation. The window information corresponding to the initial focus parameter is determined, and the window information is compared with the image size. When the window information is not entirely within the range corresponding to the image size, the initial focus parameter is adjusted so that the window information corresponding to the adjusted second target focus parameter can be entirely within the range corresponding to the image size.

[0170] For example, Figure 4 As shown in the figure, the terminal device and the camera device are connected via USB, and the image is opened through the camera APP in the terminal device. The customer sends the required second target focus parameters to the camera device through the UVC private protocol on the APP according to actual needs. The camera device obtains the clarity report of the current window based on the received second target focus parameters, and adjusts the drive motor to drive the lens to realize the focus function. Among them, the UVC private protocol can be as follows Figure 5 As shown in the figure, Cmd\Req\Value\Index\Length is defined by the UVC general protocol standard, and Data is custom data. When using it, customers can select Value\Index according to actual conditions to avoid conflicts with other camera functions.

[0171] Specifically, according to the image size and user needs, the initial focus parameters of the camera device are determined and adjusted to achieve accurate focus on the area of ​​interest. This helps to improve the clarity and quality of the image captured by the camera device, better meet user needs, and further improve user experience. It also solves the problem in the related art that the camera device cannot achieve free focus, which reduces the user's use effect, realizes the free focus of the camera device, improves the user's use effect of the camera device, and further improves user satisfaction.

[0172] The method enables the user to perform keyword detection on the initial information according to the initial information required by the terminal device when collecting the camera device, which helps to accurately identify the user's needs or intentions so as to perform focusing processing according to the user's needs in the future. By performing target detection on the first image, the target object in the image can be identified, which is helpful for subsequent image segmentation and focusing operations. According to the target information and the target keyword, the first image is segmented, and the image can be segmented into multiple second images. Then, by calculating the information matching degree between the multiple second images and the initial information, it can be determined which second image best meets the user's needs, thereby determining the target image. Finally, according to the position information of the target image, the first target focus parameter of the camera device is determined to ensure that the camera device can accurately focus on the target object according to the user's needs, realize the purpose of free focus of the user, and thus realize the user's needs or intentions. The problem that the camera device cannot achieve free focus and reduces the user's use effect in the related art is solved, and the free focus of the camera device is realized, the effect of the user using the camera device is improved, and the user satisfaction is further improved.

[0173] Corresponding to the above method embodiment, the present application also provides an embodiment of a focusing device of a camera device, Figure 6 FIG. 2 is a schematic diagram showing a focusing device 200 of a camera device according to an embodiment of the present application. Figure 6As shown, the focusing device 200 of the camera device includes: a data acquisition module 201, a data detection module 202, a target detection module 203, an image segmentation module 204, an information matching module 205, a parameter determination module 206, and a data sending module 207, wherein the data acquisition module 201 is configured to obtain a first image captured by the camera device that is connected to the terminal device for communication and obtain initial information required by the user to set the camera device capture mode through the terminal device; the data detection module 202 is configured to determine the information type corresponding to the initial information, and if the information type is a text type, perform keyword detection on the initial information to obtain the target keyword corresponding to the initial information; the target detection module 203 is configured to perform target detection on the first image according to the target keyword. Detection, to obtain target information corresponding to the first image; an image segmentation module 204, configured to perform image segmentation on the first image according to the target information and the target keyword, to obtain multiple second images; an information matching module 205, configured to respectively calculate the information matching degree between the multiple second images and the initial information, and determine the target image corresponding to the initial information from the multiple second images according to the information matching degree; a parameter determination module 206, configured to determine the position information corresponding to the target image from the first image, and determine the first target focus parameter corresponding to the camera device according to the position information; a data sending module 207, configured to send the first target focus parameter to the camera device, so that the camera device performs a focusing operation according to the first target focus parameter.

[0174] In some optional embodiments, the data detection module 202 is further configured to:

[0175] An entity recognition module is configured to perform entity recognition on the initial information using a named entity recognition model to obtain initial keywords corresponding to the initial information;

[0176] The entity unification module is configured to perform entity unification processing on the initial keyword to obtain the target keyword corresponding to the initial information.

[0177] In some optional embodiments, the target information includes a target object and a target position corresponding to the target object, and the image segmentation module is further configured to:

[0178] a relationship extraction module configured to determine a target relationship between the target objects from the initial information in combination with the target keyword using a relationship recognition model;

[0179] The segmentation processing module is configured to perform image segmentation on the first image according to the target relationship and the target position to obtain a plurality of the second images.

[0180] In some optional embodiments, the information matching module 205 is further configured to:

[0181] A feature representation module is configured to determine a first image feature corresponding to the second image and a first text feature corresponding to the initial information by using an information feature representation layer of an information matching model;

[0182] An information enhancement module is configured to use the feature enhancement layer of the information matching model to perform image information enhancement on the first image feature to obtain a second image feature and to perform text information enhancement on the first text feature to obtain a second text feature;

[0183] The semantic matching module is configured to perform semantic matching on the second image feature and the second text feature using the relevance matching layer of the information matching model to obtain the information matching degree between the second image and the initial information.

[0184] In some optional embodiments, the semantic matching module is further configured to:

[0185] a matrix acquisition module, configured to calculate the correspondence between the local area corresponding to the second image feature and the word corresponding to the second text feature by using the local matching unit of the correlation matching layer, and obtain a first mutual matrix between the local area and the word;

[0186] an image attention module, configured to determine an image attention region vector corresponding to the second text feature using the text attention unit of the correlation matching layer;

[0187] A first fusion module is configured to use the first fusion unit of the correlation matching layer to calculate the first mutual matrix and the focus image region vector to determine a first local fusion vector;

[0188] A focused text module, configured to determine a focused text region vector corresponding to the second image feature using the image attention unit of the correlation matching layer;

[0189] A second fusion module is configured to use the second fusion unit of the correlation matching layer to calculate the first mutual matrix and the focus text area vector to determine a second local fusion vector;

[0190] A first matching module is configured to fuse the first local fusion vector and the second local fusion vector by using the local fusion unit of the correlation matching layer to obtain a first matching degree corresponding to the second image feature and the second text feature;

[0191] A text representation module is configured to perform vector averaging on the focus image region vector using the global text representation unit of the correlation matching layer to obtain an average text vector, and perform linear projection on the average text vector to obtain a global text representation;

[0192] An image representation module is configured to perform vector averaging on the focus text region vector using the global image representation unit of the correlation matching layer to obtain an average image vector, and perform linear projection on the average image vector to obtain a global image representation;

[0193] A second matching module is configured to fuse the global text representation and the global image representation using the global fusion unit of the correlation matching layer to obtain a second matching degree corresponding to the second image feature and the second text feature;

[0194] The target matching module is configured to fuse the first matching degree and the second matching degree representation using the overall fusion unit of the correlation matching layer to obtain the information matching degree corresponding to the second image feature and the second text feature.

[0195] In some optional embodiments, the parameter determination module 206 is configured to:

[0196] a window determination module, configured to determine a target window corresponding to the initial information according to the position information;

[0197] A first center determination module is configured to determine a focus center corresponding to when the camera device focuses according to the target window;

[0198] The first parameter determination module is configured to determine the first target focus parameter corresponding to the camera device according to the focus center and the target window.

[0199] In some optional embodiments, the focusing device 200 of the camera device is further configured as follows:

[0200] an image determination module, configured to obtain an image size corresponding to the first image;

[0201] A second center determination module is configured to determine a trigger center corresponding to the user in the terminal device according to the initial information;

[0202] A size determination module, configured to determine the window size of the focus window when the camera device performs focusing;

[0203] A second parameter determination module is configured to determine an initial focus parameter corresponding to the camera device according to the trigger center and the window size;

[0204] The parameter adjustment module is configured to adjust the initial focus parameter according to the image size to obtain a second target focus parameter.

[0205] The above is a schematic scheme of a focusing device 200 of a camera device of the present embodiment. It should be noted that the technical scheme of the focusing device 200 of the camera device and the technical scheme of the focusing method of the camera device described above belong to the same concept, and the details of the technical scheme of the focusing device of the camera device that are not described in detail can all be referred to the description of the technical scheme of the focusing method of the camera device described above.

[0206] like Figure 7 As shown, Figure 7 The structure block diagram of a terminal device 300 provided according to an embodiment of the present application is shown. The components of the terminal device 300 include but are not limited to a memory 310 and a processor 320. The processor 320 is connected to the memory 310 via a bus 330, and the database 350 is used to store data.

[0207] The terminal device 300 also includes an access device 340, which enables the terminal device 300 to communicate via one or more networks 360. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 340 may include one or more of any type of network interface (e.g., a network interface card (NIC)) of wired or wireless, such as an IEEE802.11 wireless local area network (WLAN) wireless interface, a global interconnection for microwave access (Wi-MAX) interface, an Ethernet interface, a universal serial bus (USB) interface, a cellular network interface, a Bluetooth interface, a near field communication (NFC) interface, and the like.

[0208] In one embodiment of the present application, the above components of the terminal device 300 and Figure 7 Other components not shown in the figure may also be connected to each other, for example, via a bus. It should be understood that Figure 7 The electronic device structure block diagram shown is only for the purpose of illustration, and is not intended to limit the scope of the present application. Those skilled in the art may add or replace other components as needed.

[0209] The terminal device 300 may be any type of stationary or mobile electronic device, including a mobile computer or mobile electronic device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smart phone), a wearable electronic device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary electronic device such as a desktop computer or a PC. The terminal device 300 may also be a mobile or stationary server.

[0210] The processor 320 is used to execute computer executable instructions of the object processing method.

[0211] The above is a schematic scheme of a computing device of this embodiment. It should be noted that the technical scheme of the computing device and the technical scheme of the object processing method described above are of the same concept, and the details not described in detail in the technical scheme of the computing device can be referred to the description of the technical scheme of the object processing method described above.

[0212] An embodiment of the present application further provides a storage medium, which is a computer-readable storage medium and stores a computer program. When the computer program is executed by a processor, the focusing method of the above-mentioned camera device is implemented.

[0213] As a non-transient computer-readable storage medium, the memory can be used to store non-transient software programs and non-transient computer executable programs. In addition, the memory may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage devices. In some embodiments, the memory may optionally include a memory remotely arranged relative to the processor, and these remote memories may be connected to the processor via a network. Examples of the above-mentioned networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof. The device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and are implemented to be located in one place, or may also be distributed to multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment.

[0214] It will be appreciated by those skilled in the art that all or some of the steps and systems in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or as hardware, or as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or non-transitory medium) and a communication medium (or transient medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules, or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disk (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media generally include computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0215] The above is a specific description of the preferred implementation of the present application, but the present application is not limited to the above implementation mode. Technical personnel familiar with the field can also make various equivalent modifications or substitutions under the shared conditions without violating the spirit of the present application. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present application.

Claims

1. A focusing method for a camera device, characterized in that: The method comprises: Obtaining a first image captured by the camera device that is communicatively connected to the terminal device and obtaining initial information required by a user to set a capture mode of the camera device through the terminal device; Determine the information type corresponding to the initial information, and if the information type is a text type, perform keyword detection on the initial information to obtain a target keyword corresponding to the initial information; Performing target detection on the first image according to the target keyword to obtain target information corresponding to the first image; The first image is segmented according to the target information and the target keyword to obtain a plurality of second images, wherein the target information includes a target object and a target position corresponding to the target object; specifically comprising: using a relationship recognition model to determine a target relationship between the target objects in combination with the target keyword from the initial information; and the first image is segmented according to the target relationship and the target position to obtain a plurality of second images; respectively calculating information matching degrees between a plurality of the second images and the initial information; Determining a target image corresponding to the initial information from a plurality of second images according to the information matching degree; The respectively calculating the information matching degree between the plurality of second images and the initial information specifically comprises: using the information feature representation layer of the information matching model to determine the first image feature corresponding to the second image and the first text feature corresponding to the initial information; using the feature enhancement layer of the information matching model to perform image information enhancement on the first image feature to obtain the second image feature and to perform text information enhancement on the first text feature to obtain the second text feature; using the correlation matching layer of the information matching model to perform semantic matching on the second image feature and the second text feature to obtain the information matching degree between the second image and the initial information; The method of using the correlation matching layer of the information matching model to semantically match the second image feature and the second text feature to obtain the information matching degree between the second image and the initial information specifically includes: using the local matching unit of the correlation matching layer to calculate the correspondence between the local area corresponding to the second image feature and the word corresponding to the second text feature to obtain a first mutual matrix between the local area and the word; using the text attention unit of the correlation matching layer to determine the image area vector corresponding to the second text feature; using the first fusion unit of the correlation matching layer to calculate the first mutual matrix and the image area vector to determine a first local fusion vector; using the image attention unit of the correlation matching layer to determine the text area vector corresponding to the second image feature; using the second fusion unit of the correlation matching layer to calculate the first mutual matrix and the text area vector to determine a second local fusion vector; using the correlation The local fusion unit of the matching layer fuses the first local fusion vector and the second local fusion vector to obtain the first matching degree corresponding to the second image feature and the second text feature; the global text representation unit of the correlation matching layer performs vector averaging on the vectors of the focus image region to obtain an average text vector, and linearly projects the average text vector to obtain a global text representation; the global image representation unit of the correlation matching layer performs vector averaging on the vectors of the focus text region to obtain an average image vector, and linearly projects the average image vector to obtain a global image representation; the global fusion unit of the correlation matching layer fuses the global text representation and the global image representation to obtain the second matching degree corresponding to the second image feature and the second text feature; the overall fusion unit of the correlation matching layer fuses the first matching degree and the second matching degree to obtain the information matching degree corresponding to the second image feature and the second text feature; Determine position information corresponding to the target image from the first image, and determine a first target focus parameter corresponding to the camera device according to the position information; The first target focus parameter is sent to the camera device, so that the camera device performs a focus operation according to the first target focus parameter.

2. The method according to claim 1, characterized in that The performing keyword detection on the initial information to obtain a target keyword corresponding to the initial information includes: Performing entity recognition on the initial information using a named entity recognition model to obtain initial keywords corresponding to the initial information; The initial keywords are subjected to entity uniform processing to obtain the target keywords corresponding to the initial information.

3. The method according to claim 1, characterized in that The determining, according to the position information, a first target focus parameter corresponding to the camera device includes: Determine a target window corresponding to the initial information according to the position information; Determining, according to the target window, a focus center corresponding to when the camera device performs focusing; The first target focus parameter corresponding to the camera device is determined according to the focus center and the target window.

4. The method according to any one of claims 1 to 3, characterized in that If the information type is an input instruction type, the method further includes: Obtaining an image size corresponding to the first image; Determine, according to the initial information, a trigger center corresponding to the user in the terminal device; Determining the window size of the focus window when the camera device performs focusing; Determine the initial focus parameter corresponding to the camera device according to the trigger center and the window size; The initial focus parameter is adjusted according to the image size to obtain a second target focus parameter.

5. A focusing device for a camera device, characterized in that: include: A data acquisition module is configured to obtain a first image captured by the camera device that is communicatively connected to the terminal device and to obtain initial information required by a user to set a capture mode of the camera device through the terminal device; A data detection module is configured to determine the information type corresponding to the initial information, and if the information type is a text type, perform keyword detection on the initial information to obtain a target keyword corresponding to the initial information; a target detection module, configured to perform target detection on the first image according to the target keyword to obtain target information corresponding to the first image; The image segmentation module is configured to perform image segmentation on the first image according to the target information and the target keyword to obtain a plurality of second images, wherein the target information includes a target object and a target position corresponding to the target object; specifically comprising: using a relationship recognition model to determine a target relationship between the target objects from the initial information in combination with the target keyword; performing image segmentation on the first image according to the target relationship and the target position to obtain a plurality of second images; an information matching module, configured to respectively calculate information matching degrees between the plurality of second images and the initial information; and determine a target image corresponding to the initial information from the plurality of second images according to the information matching degrees; The information matching degree between the plurality of second images and the initial information is calculated respectively, specifically comprising: determining the first image feature corresponding to the second image and the first text feature corresponding to the initial information by using the information feature representation layer of the information matching model; performing image information enhancement on the first image feature to obtain the second image feature and performing text information enhancement on the first text feature to obtain the second text feature by using the feature enhancement layer of the information matching model; performing semantic matching on the second image feature and the second text feature by using the relevance matching layer of the information matching model to obtain the information matching degree between the second image and the initial information; performing semantic matching on the second image feature and the second text feature by using the relevance matching layer of the information matching model to obtain the information matching degree between the second image and the initial information specifically comprises: calculating the correspondence between the local area corresponding to the second image feature and the word corresponding to the second text feature by using the local matching unit of the relevance matching layer to obtain the first mutual matrix between the local area and the word; determining the attention image area vector corresponding to the second text feature by using the text attention unit of the relevance matching layer; calculating the first mutual matrix by using the first fusion unit of the relevance matching layer, and calculating the first mutual matrix and the attention image area vector. Determine a first local fusion vector; determine the attention text area vector corresponding to the second image feature using the image attention unit of the correlation matching layer; calculate the first mutual matrix using the second fusion unit of the correlation matching layer, and determine the second local fusion vector after calculating the first mutual matrix and the attention text area vector; fuse the first local fusion vector and the second local fusion vector using the local fusion unit of the correlation matching layer to obtain a first matching degree corresponding to the second image feature and the second text feature; perform vector averaging on the attention image area vector using the global text representation unit of the correlation matching layer to obtain an average text vector, and perform linear projection on the average text vector to obtain a global text representation; perform vector averaging on the attention text area vector using the global image representation unit of the correlation matching layer to obtain an average image vector, and perform linear projection on the average image vector to obtain a global image representation; fuse the global text representation and the global image representation using the global fusion unit of the correlation matching layer to obtain a second matching degree corresponding to the second image feature and the second text feature; fuse the first matching degree and the second matching degree using the overall fusion unit of the correlation matching layer to obtain the information matching degree corresponding to the second image feature and the second text feature; a parameter determination module, configured to determine position information corresponding to the target image from the first image, and determine a first target focus parameter corresponding to the camera device according to the position information; The data sending module is configured to send the first target focus parameter to the camera device, so that the camera device performs a focus operation according to the first target focus parameter.

6. A terminal device, characterized in that: The terminal device includes at least one control processor and a memory for communicating with the at least one control processor; the memory stores instructions that can be executed by the at least one control processor, and the instructions are executed by the at least one control processor so that the at least one control processor can execute the focusing method of the camera device as described in any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and the computer-executable instructions are used to enable a computer to execute the steps of the focusing method of the camera apparatus according to any one of claims 1 to 4.

Images