AI user behavior learning method based on Tof focusing
Through Tof focus technology and AI user behavior learning, the Tof focus model is generated, which solves the problem of poor autofocus effect of infrared cameras under different scenes and user needs, and realizes personalized autofocus, improving shooting effect and user experience.
Patent Information
- Application Number
- CN202411555864.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-04
- Publication Date
- 2024-12-03
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The existing infrared cameras have poor autofocus function under different shooting scenes and user needs, which affects the user experience.
Using the AI user behavior learning method based on Tof focus, the distance of objects in the environment scene is measured through the Tof sensing unit of the image acquisition device, combined with the historical image data and user focus point in the image database, the Tof focus model is generated to achieve automatic Focus.
It realizes automatic adjustment of focus points according to user shooting behavior, meets the personalized shooting needs of different users, and improves shooting effect and user experience.
Smart Images

Figure CN119071632A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of image processing technology, and in particular to an AI user behavior learning method based on Tof focusing. Background Art
[0002] A camera is a device that uses the principle of optical imaging to form an image and uses film to record the image. It is an optical instrument used for photography. In modern life, there are many devices that can record images, and they all have the characteristics of a camera. After the light reflected by the subject is focused through the camera lens and the shutter that controls the exposure, the subject forms a latent image on the photosensitive material in the darkroom, which forms a permanent image after washing. Focusing refers to adjusting the focal distance when using the camera. In related technologies, ordinary infrared cameras with continuous zoom often only have an autofocus mode. When this type of infrared camera is working normally, when using the autofocus function to shoot an object, the shooting effect is often poor, and the focus object cannot adapt to different shooting scenes and the usage needs of different users, which in turn affects the user's product experience. Summary of the invention
[0003] In order to overcome the problems existing in the related art, the present disclosure provides an AI user behavior learning method based on Tof focusing.
[0004] According to a first aspect of an embodiment of the present disclosure, there is provided an AI user behavior learning method based on Tof focusing, which is applied to an image acquisition device, and the method includes: In response to the image acquisition of the current environment scene by the camera device corresponding to the image acquisition device, scene image data corresponding to the current environment scene is acquired; Based on the time-of-flight Tof sensing unit corresponding to the camera device, a preset beam is emitted to the current environment scene, and Tof distance data corresponding to the scene image data is determined through the reflected beam corresponding to the object in the current environment scene, wherein the Tof distance data is used to indicate the visual distance between each object in the current environment scene and the camera device; Determine the first RGB pixel value distribution data corresponding to the scene image data, and obtain, from an image database corresponding to the image acquisition device, multiple frames of historical image data matching the first RGB pixel value distribution data, as well as multiple Tof historical distance data corresponding to the multiple frames of historical image data, and multiple Tof focus points corresponding to the multiple frames of historical image data, wherein each Tof focus point is a focus point selected by a user in the corresponding historical image; According to the multiple Tof historical distance data and the multiple Tof focus points, the user's shooting focus behavior based on the image acquisition device is learned, and the parameters of the preset AI model are updated based on the learning result to generate a Tof focus model; The Tof distance data is identified based on the Tof focusing model, a target Tof focusing point position corresponding to the scene image data is generated, and the image acquisition device is automatically focused based on the target Tof focusing point position.
[0005] Optionally, acquiring a plurality of frames of historical image data matching the first RGB pixel value distribution data from an image database corresponding to the image acquisition device includes: Determine a plurality of second RGB pixel value distribution data corresponding one to one to a plurality of frames of stored image data in the image database; Based on the plurality of second RGB pixel value distribution data and the first RGB pixel value distribution data, comparing the scene image data with the plurality of frames of stored image data frame by frame and pixel by pixel to generate a plurality of similarity indexes; According to the multiple similarity indexes, a plurality of frames of stored images having the largest similarity index with the scene image data are obtained from the plurality of frames of historical image data as the plurality of frames of historical image data.
[0006] Optionally, the comparing the scene image data with the multiple frames of stored image data frame by frame and pixel by pixel based on the multiple second RGB pixel value distribution data and the first RGB pixel value distribution data to generate multiple similarity indexes includes: Determining a first brightness comparison value, a first contrast comparison value, and a first structure comparison value between the scene image data and the first stored image data according to the third RGB pixel value distribution data and the first RGB pixel value distribution data, wherein the first stored image data is any one of the multiple frames of stored image data, and the third RGB pixel value distribution data is the RGB pixel value corresponding to the first stored image data; According to the first brightness comparison value, the first contrast comparison value, and the first structure comparison value, a first similarity index between the first stored image data and the scene image data is determined based on the following formula:
[0007] Among them, the is the X-axis component of the first brightness comparison value, is the Y-axis component of the first brightness comparison value, is the X-axis component of the first contrast comparison value, is the Y-axis component of the first contrast comparison value, is the first structure comparison value, the is the first constant, is the second constant.
[0008] Optionally, the comparing the scene image data with the multiple frames of stored image data frame by frame and pixel by pixel based on the multiple second RGB pixel value distribution data and the first RGB pixel value distribution data to generate multiple similarity indexes includes: Determining a local window size for image segmentation according to the image resolution of the multiple frames of stored image data, wherein the multiple local windows segment any stored image into multiple local images of the same size; Segmenting the second stored image data according to the multiple local windows to generate multiple frames of local image data, wherein the second stored image data is any of the multiple frames of stored image data; determining a plurality of sub-similarity indexes according to a third RGB pixel value and the first RGB pixel value, the third RGB pixel value being a pixel value corresponding to the second stored image data; Averaging the plurality of sub-similarity indexes to generate a first similarity index between the second stored image data and the scene image data; Based on the above steps, the plurality of frames of stored image data are traversed to generate the plurality of similarity indexes.
[0009] Optionally, the step of learning the user's shooting focus behavior based on the image acquisition device according to the multiple Tof historical distance data and the multiple Tof focus points, and updating the parameters of the preset AI model based on the learning result to generate a Tof focus model includes: Based on the preset AI model, according to the multiple Tof historical distance data and the corresponding multiple Tof focus point positions, the user's shooting focus behavior analysis is performed to generate the user's focus behavior data; Determining the user personality preference data according to the multiple frames of historical image data and the focusing behavior data, wherein the user personality preference data is used to indicate the user's preference for shooting focusing behavior; Extracting features from the user's personality preference data using a preset feature algorithm to generate user feature preference data for the user; The parameters of the preset AI model are updated based on the user feature preference data to generate the Tof focusing model.
[0010] Optionally, extracting features from the user personality preference data by a preset feature algorithm to generate user feature preference data of the user includes: Identify physical features in first historical image data to generate item category parameters of the first historical image data, where the first historical image data is any one of the multiple frames of historical image data; Determine the target item category corresponding to the first historical image data according to the position of the item category parameter in the first historical image data and the first Tof focus point position corresponding to the first historical image data; According to the item category parameter and the target item category, generating a preference feature vector corresponding to the first historical image data by using the preset feature algorithm; generating the first user characteristic preference data according to the preference characteristic vector; Traversing the multiple frames of historical image data to generate multiple user feature preference data; The plurality of user characteristic preference data are weighted averaged to generate the user characteristic preference data.
[0011] Optionally, the identifying the Tof distance data based on the Tof focus model to generate a target Tof focus point position corresponding to the scene image data includes: Identify the Tof distance data based on the Tof focus model, and generate a focus weight corresponding to the scene image data; Identify the target focused object in the scene image data according to the focus weight, and determine the two-dimensional coordinate range of the target focused object in the scene image data; The target Tof focus point position is determined according to the Tof distance data and the two-dimensional coordinate range.
[0012] According to a second aspect of an embodiment of the present disclosure, there is provided an AI user behavior learning device based on Tof focusing, which is applied to an image acquisition device, and the device includes: A collection module, configured to acquire scene image data corresponding to the current environment scene in response to image collection of the current environment scene by a camera device corresponding to the image collection device; A determination module is configured to transmit a preset beam to the current environment scene based on a time-of-flight Tof sensing unit corresponding to the camera device, and determine Tof distance data corresponding to the scene image data through a reflected beam corresponding to an object in the current environment scene, wherein the Tof distance data is used to indicate a visual distance between each object in the current environment scene and the camera device; an acquisition module, configured to determine the first RGB pixel value distribution data corresponding to the scene image data, and acquire, from an image database corresponding to the image acquisition device, a plurality of frames of historical image data matching the first RGB pixel value distribution data, and a plurality of Tof historical distance data corresponding one-to-one to the plurality of frames of historical image data, and a plurality of Tof focus points corresponding one-to-one to the plurality of frames of historical image data, wherein each Tof focus point position is a focus point position selected by a user in the corresponding historical image; A generation module is configured to learn the user's shooting focus behavior based on the image acquisition device according to the multiple Tof historical distance data and the multiple Tof focus point positions, and update the parameters of the preset AI model based on the learning result to generate a Tof focus model; An execution module is configured to identify the Tof distance data based on the Tof focus model, generate a target Tof focus point position corresponding to the scene image data, and automatically focus the image acquisition device based on the target Tof focus point position.
[0013] According to a third aspect of an embodiment of the present disclosure, there is provided a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, the steps of any one of the methods described in the first aspect are implemented.
[0014] According to a fourth aspect of an embodiment of the present disclosure, there is provided an electronic device, including: processor; a memory for storing processor-executable instructions; The processor is configured to execute executable instructions in the memory to implement the method described in any one of the first aspects.
[0015] The technical solution provided by the embodiments of the present disclosure may have the following beneficial effects: Through the above-mentioned method, in response to the image acquisition of the current environment scene by the camera device corresponding to the image acquisition device, the scene image data corresponding to the current environment scene is obtained, and the time-of-flight Tof sensing unit corresponding to the camera device transmits a preset beam to the current environment scene, and the Tof distance data corresponding to the scene image data is determined through the reflected beam corresponding to the object in the current environment scene, wherein the Tof distance data is used to indicate the visual distance between each object in the current environment scene and the camera device, and the first RGB pixel value distribution data corresponding to the scene image data is determined, and a multi-frame history image matching the first RGB pixel value distribution data is obtained from the image database corresponding to the image acquisition device. Image data, as well as multiple Tof historical distance data corresponding to multiple frames of historical image data, and multiple Tof focus points corresponding to multiple frames of historical image data, wherein each Tof focus point is a focus point selected by the user in the corresponding historical image. According to multiple Tof historical distance data and multiple Tof focus points, the user's shooting focus behavior based on the image acquisition device is learned, and the parameters of the preset AI model are updated based on the learning results to generate a Tof focus model, and the Tof distance data is identified based on the Tof focus model to generate the target Tof focus point corresponding to the scene image data, and the image acquisition device is automatically focused based on the target Tof focus point. In this way, the current shooting scene is automatically focused based on the user's shooting behavior, meeting the personalized focus shooting needs of different users.
[0016] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.
[0018] Figure 1 It is a flowchart of an AI user behavior learning method based on Tof focusing according to an exemplary embodiment.
[0019] Figure 2 It is a block diagram of an AI user behavior learning device based on Tof focusing according to an exemplary embodiment. DETAILED DESCRIPTION
[0020] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.
[0021] It should be noted that all actions of acquiring signals, information or data in this application are carried out in compliance with the relevant data protection laws and policies of the country where they are located and with the authorization given by the owner of the corresponding device.
[0022] Figure 1 is a flow chart of an AI user behavior learning method based on Tof focusing according to an exemplary embodiment. Figure 1 As shown, the method is applied to an image acquisition device, and the method comprises: In step S11, in response to the image acquisition device corresponding to the camera apparatus performing image acquisition on the current environment scene, scene image data corresponding to the current environment scene is acquired.
[0023] It should be noted that, in the process of capturing images of the current environment scene based on the image acquisition device, the user will select a close-up object or a distant object in the image for focus based on the image displayed in real time in the image acquisition device. The image acquisition device uses a preset focus algorithm to adjust the focus distance and focus based on the focus point clicked by the user on the real-time displayed image. In most cases, when the user holds the image acquisition device for image acquisition, it is likely that the operator's hand is occupied and the focus point cannot be selected, or the focus point is improperly selected, resulting in blurred scene image data generated by the shooting. Therefore, this embodiment provides an AI user behavior learning method based on Tof focusing to realize automatic detection of focus points in different shooting scenes based on the user's shooting focusing behavior habits, and automatic focusing of the image acquisition device based on the generated focus point.
[0024] For example, when a user captures an image of the current environment scene based on the camera device corresponding to the image acquisition device, the scene image data corresponding to the current environment scene is obtained. In this embodiment, when the user turns on the image acquisition device to capture an image, it is determined that the current user is capturing the current environment image based on the shooting device corresponding to the image acquisition device. In some embodiments, the image acquisition device can be a terminal. When it is detected that the user opens the shooting APP of the terminal, it is determined that the current user's entire machine and the camera device are capturing the image of the current environment scene.
[0025] Among them, the scene image data is the real-time image data of the current environment scene captured by the image acquisition device based on the camera device. The real-time image data will be based on the current environment scene, corresponding to the shooting angle, shooting distance and shooting object of the camera device, and different real-time scene image data will be displayed in the display area of the image acquisition device.
[0026] In step S12, a preset beam is emitted to the current environment scene based on the time-of-flight Tof sensing unit corresponding to the camera device, and the Tof distance data corresponding to the scene image data is determined through the reflected beam corresponding to the object in the current environment scene, wherein the Tof distance data is used to indicate the visual distance between each object in the current environment scene and the camera device.
[0027] For example, the image acquisition device of this embodiment is equipped with a Tof sensor unit, based on which the distance of each object in the current environment scene is identified, thereby determining the Tof distance data corresponding to the scene image data. The Tof distance data is used to indicate the visual distance between each object in the current environment scene and the camera device. Tof focusing is a technology that uses the time of flight principle to measure distance, and is widely used in photography, 3D imaging, autonomous driving vehicles, robot navigation and other fields. In the field of photography and imaging, Tof focusing technology can help the camera focus quickly and accurately, improving shooting efficiency and photo quality. For example, the working principle of Tof focusing is: (1) transmitting signal: the Tof camera will emit a beam of infrared light (or other types of electromagnetic waves); (2) measuring time: this beam of light will irradiate the object corresponding to the current environmental scene and then reflect back. The camera measures the time it takes for the light beam to be emitted and reflected back; (3) Calculate the distance: Based on the speed of light and the measured time, calculate the round-trip distance of the light beam, that is, the distance between the camera and the object; (4) Adjust the focal length: Based on the calculated distance, the camera automatically adjusts the focal length of the lens to achieve clear imaging.
[0028] In this embodiment, the Tof focusing technology is used to focus the image acquisition device. The advantage of Tof focusing is that it can focus quickly. The Tof technology can measure the distance very quickly and achieve fast focusing. And the adaptability of the Tof focusing technology is strong. Under different lighting conditions, the Tof focusing can maintain good performance. Moreover, the precision of the Tof focusing technology is high. Compared with the traditional phase focusing or contrast focusing, the Tof focusing can provide higher precision. The image acquisition device of this embodiment is equipped with the Tof focusing technology, which transmits a preset beam to the current environment scene based on the corresponding Tof sensing unit. The preset beam can be infrared light. The Tof distance data corresponding to the scene image data currently captured is determined by the reflected beam corresponding to the object in the current environment scene. The Tof distance data is used to indicate the visual distance of different objects in the current environment scene relative to the camera device.
[0029] It should be noted that the AI user behavior learning method based on Tof focusing corresponding to the present embodiment is suitable for environmental scenes where automatic focusing can be performed based on Tof. The image acquisition device identifies the Tof distance data of each object in the current environmental scene based on the Tof sensing unit, thereby realizing automatic focusing of the scene image data in the current environmental scene according to the Tof distance data.
[0030] In step S13, the first RGB pixel value distribution data corresponding to the scene image data is confirmed, and multiple frames of historical image data matching the first RGB pixel value distribution data, as well as multiple Tof historical distance data corresponding one-to-one to the multiple frames of historical image data, and multiple Tof focus point positions corresponding one-to-one to the multiple frames of historical image data are obtained from the image database corresponding to the image acquisition device, wherein each Tof focus point position is a focus point position selected by the user in the corresponding historical image.
[0031] For example, the image acquisition device calculates and determines the distribution data of the first RGB pixel value corresponding to the current scene image data. In this embodiment, in order to avoid calculation redundancy, when it is detected that the current environmental scene has not changed for a set time, the RGB pixel value corresponding to the scene image data is analyzed based on the above steps to obtain the RGB pixel value corresponding to the scene image data. Among them, the RGB pixel value distribution data is used to indicate the distribution of pixel values of pixels at each position in the scene image data, and the image acquisition device can determine the similarity between the shooting scenes corresponding to each image frame according to the distribution of RGB pixel values between each image frame. In this embodiment, based on the first RGB pixel value distribution data corresponding to the scene image data, multiple frames of historical image data matching the first RGB pixel value distribution data are obtained from the corresponding image database. Among them, if the average value of the difference between the RGB pixel value of each pixel corresponding to the first historical image data in the image database and the RGB pixel value of the pixel at the same position corresponding to the scene image data is less than the set threshold, it is determined that the shooting scene corresponding to the first historical image data is similar to the scene image data, and the user focusing behavior of the scene image data can be analyzed based on the first historical image data.
[0032] In this embodiment, based on the first RGB pixel value distribution data and the RGB pixel value distribution data corresponding one to one with the image data stored in the image database, the scene image data is compared with the stored image data. When the sum of the RGB pixel value differences at the same position is less than the set threshold value, it is determined that the currently compared stored image data is the historical image data that matches the first RGB pixel value distribution data. In this way, multiple frames of historical image data are determined from the image database, and multiple Tof historical distance data of objects in the corresponding environmental scene when the image is taken are determined based on the multiple frames of historical image data, as well as multiple Tof focus points corresponding to the multiple frames of historical image data. It should be noted that the Tof focus point position in this embodiment is the Tof focus point position selected by the user manually clicking on the real-time image when generating the corresponding historical image data, or the Tof focus point position automatically determined by the image acquisition device based on the corresponding focusing algorithm.
[0033] In this embodiment, the Tof focus point selected by the user when taking images in the historical image data is analyzed to learn the user's shooting behavior when using the image acquisition device, and based on the shooting behavior, Tof automatic focusing is performed on the scene image data corresponding to the current environmental scene to achieve personalized focus shooting needs of different users in different scenes.
[0034] Optionally, in some embodiments, the above step S13 includes the following steps: In step S131, a plurality of second RGB pixel value distribution data corresponding one to one to multiple frames of stored image data in an image database is determined.
[0035] For example, in this embodiment, multiple frames of stored image data are stored in the image database. When the image acquisition device acquires images of the current environment and generates scene image data, the generated stored image data and the corresponding RGB pixel value distribution data, as well as the Tof historical distance data corresponding to the stored image data are associated and saved to obtain multiple frames of stored image data corresponding to the second RGB pixel value distribution data, and multiple frames of stored image data corresponding to the Tof historical distance data.
[0036] In step S132, based on the plurality of second RGB pixel value distribution data and the first RGB pixel value distribution data, the scene image data is compared with the plurality of frames of stored image data frame by frame and pixel by pixel to generate a plurality of similarity indexes.
[0037] For example, in this embodiment, the scene image data corresponding to the current environment scene is compared one by one with the multi-frame stored image data stored in the image database, and the similarity between the scene image data and the multi-frame stored image data is determined based on the multiple second RGB pixel value distribution data corresponding to the multi-frame stored image data and the first RGB pixel value distribution data corresponding to the scene image data to generate multiple similarity indexes.
[0038] Optionally, in some embodiments, the above step S132 includes: Determine a first brightness comparison value, a first contrast comparison value, and a first structure comparison value between the scene image data and the first stored image data according to the third RGB pixel value distribution data and the first RGB pixel value distribution data, wherein the first stored image data is any multi-frame stored image data, and the third RGB pixel value distribution data is the RGB pixel value corresponding to the first stored image data; According to the first brightness comparison value, the first contrast comparison value and the first structure comparison value, a first similarity index between the first stored image data and the scene image data is determined based on the following formula:
[0039] in, is the X-axis component of the first brightness comparison value, is the Y-axis component of the first brightness comparison value, is the X-axis component of the first contrast comparison value, is the Y-axis component of the first contrast comparison value, is the first structure comparison value, is the first constant, is the second constant.
[0040] For example, in this embodiment, a structural similarity index calculation method is used to determine the similarity between the scene image data and the first stored image data, where the first stored image data is any multi-frame stored image data. The structural similarity index SSIM is an image quality assessment method used to measure the visual similarity between two images. SSIM not only considers the brightness and contrast of the image, but also the structural information of the image, so it can better reflect the human eye's perception of image quality.
[0041] In this embodiment, the first brightness comparison value, the first contrast comparison value and the first structure comparison value between the scene image data and the first stored image data are determined based on the third RGB pixel value distribution data and the first RGB pixel value distribution data, wherein the third RGB pixel value distribution data is the RGB pixel value distribution corresponding to the first stored image data, the first brightness comparison value is the brightness mean of the two images, the first contrast comparison value is the contrast of the two images, that is, the standard deviation of the brightness, and the first structure comparison value is the brightness covariance value of the two images. After determining the above parameters, the first similarity index between the first stored image data and the scene image data is determined based on the following formula:
[0042] in, is the X-axis component of the first brightness comparison value, is the Y-axis component of the first brightness comparison value, is the X-axis component of the first contrast comparison value, is the Y-axis component of the first contrast comparison value, is the first structure comparison value, is the first constant, is the second constant. For example, and is a constant used to avoid the denominator being zero, usually , , where L and C are dynamic ranges, and k is a decimal close to 1. The parameters L, C, and k can be set based on the different focusing accuracies of the image acquisition device in different scenes.
[0043] Optionally, in some embodiments, the above step S132 includes: Determining the size of a local window for image segmentation according to the image resolution of the multiple frames of stored image data, wherein the multiple local windows segment any stored image into multiple local images of the same size; Segmenting the second stored image data according to the multiple local windows to generate multiple frames of local image data, wherein the second stored image data is any multiple frames of stored image data; determining a plurality of sub-similarity indexes according to a third RGB pixel value and the first RGB pixel value, the third RGB pixel value being a pixel value corresponding to the second stored image data; averaging the plurality of sub-similarity indexes to generate a first similarity index between the second stored image data and the scene image data; Based on the above steps, multiple frames of stored image data are traversed to generate multiple similarity indexes.
[0044] For example, in this embodiment, in order to ensure the accuracy of the similarity index, a partition comparison method is used to determine the similarity between the second stored image data and the scene image data. It should be noted that the image size, image resolution, image display orientation and other acquisition parameters between the multiple frames of stored image data and the scene image data based on the same image acquisition device are the same. According to the image resolution of the stored image data, the local window size of the image segmentation is determined, and the second stored image is segmented according to multiple local windows to generate multiple frames of local image data. Based on the processing method of the stored image, the scene image data is segmented with the same size to obtain multiple frames of sub-images. According to the third RGB pixel value distribution data and the first RGB pixel value distribution data, multiple sub-similarity indexes in different partitions are determined. The multiple sub-similarity indexes are averaged to generate a first similarity index between the second stored image data and the scene image data. Repeat the above steps to obtain multiple similarity indexes between the scene image data and the multiple frames of stored image data.
[0045] It should be noted that SSIM may not be very sensitive to global changes in the image, so this embodiment uses a local window comparison method to determine the similarity index between the stored image and the scene image to obtain a more accurate similarity index.
[0046] In step S133, based on the multiple similarity indexes, multiple frames of stored images having the largest similarity index with the scene image data are obtained from the multiple frames of historical image data as the multiple frames of historical image data.
[0047] For example, according to multiple similarity indications, multiple frames of stored images with the largest similarity index with the scene image data are obtained from multiple frames of historical image data as multiple frames of historical image data. In this embodiment, according to the similarity index corresponding to each stored image, multiple frames of historical images are sorted in descending order of similarity index according to the size relationship of the similarity index to generate a historical image sequence. The first N frames of historical images in the historical image sequence are selected as the multiple frames of subordinate image data.
[0048] In step S14, based on multiple Tof historical distance data and multiple Tof focus points, the user's shooting focus behavior based on the image acquisition device is learned, and the parameters of the preset AI model are updated based on the learning results to generate a Tof focus model.
[0049] For example, in this embodiment, based on multiple Tof historical distance data and multiple Tof focus points corresponding to each other, the user's shooting and focusing behavior based on the image acquisition device is learned, and the parameters of the preset AI model are updated based on the learning results to generate a Tof focus model. It should be noted that in this embodiment, multiple Tof historical distance data and multiple Tof focus points corresponding to each other are used as model training data sets to train the preset AI model, so that the preset AI model learns the shooting and focusing behavior habits of the current user, and updates the model parameters of the preset AI model according to the learning results to generate a Tof focus model. In this embodiment, the Tof focus model is used to be based on the Tof distance data of the current environmental scene, and the user's shooting behavior habits obtained by the above learning results are applied to determine the Tof focus point position in the current environmental scene from the Tof distance data, so as to complete the automatic focus of the image acquisition device based on the Tof focus point position.
[0050] Optionally, in some embodiments, the above-mentioned annotation S14 includes: In step S141, based on a preset AI model and according to a plurality of Tof historical distance data and a corresponding plurality of Tof focus point positions, the user's shooting focus behavior is analyzed to generate the user's focusing behavior data.
[0051] For example, in this embodiment, multiple Tof historical distance data and multiple Tof focus points are used as training data sets to train the preset AI model, so that the preset AI model learns the user's shooting focus behavior and generates the user's focusing behavior data.
[0052] In step S142, user personality preference data is determined based on the multiple frames of historical image data and the focusing behavior data, where the user personality preference data is used to indicate the user's preference for shooting focusing behavior.
[0053] The user's individual preferences are determined based on multiple frames of historical image data and focusing behavior data. The user's individual preferences are preferences for different focused objects selected by the user in different environmental scenarios.
[0054] In step S143, feature extraction is performed on the user's personality preference data using a preset feature algorithm to generate user feature preference data of the user.
[0055] For example, the user's personality preference is extracted through a preset feature algorithm to generate user feature preference data of the user.
[0056] Optionally, in some embodiments, the above step S143 includes: Identify physical features in the first historical image data to generate item category parameters of the first historical image data, where the first historical image data is any multiple frames of historical image data; Determine the target item category corresponding to the first historical image data according to the position of the item category parameter in the first historical image data and the first Tof focus point position corresponding to the first historical image data; Generate a preference feature vector corresponding to the first historical image data through a preset feature algorithm according to the item category parameter and the target item category; Generating first user characteristic preference data according to the preference characteristic vector; Traversing multiple frames of historical image data to generate multiple user feature preference data; A weighted average is performed on multiple user characteristic preference data to generate user characteristic preference data.
[0057] In step S144, the parameters of the preset AI model are updated based on the user feature preference data to generate a Tof focus model.
[0058] For example, in this embodiment, user feature preference data is used as the loss function of the preset AI model, and the parameters of the preset AI model are updated according to the loss function to generate a Tof focusing model.
[0059] In step S15, the Tof distance data is identified based on the Tof focus model, a target Tof focus point position corresponding to the scene image data is generated, and the image acquisition device is automatically focused based on the target Tof focus point position.
[0060] For example, in this embodiment, the Tof distance data is identified based on the Tof focus model, the user shooting behavior habits learned in the Tof focus model are applied, the target Tof focus point position corresponding to the scene image data is selected from the Tof distance data, and the image acquisition device is automatically focused based on the target Tof focus point position. In this embodiment, after the Tof focus point position is determined, the automatic focus point position is displayed in the currently displayed scene image to facilitate the user to know the focus point position during the image acquisition process.
[0061] Optionally, in some embodiments, the above step S15 includes: Identify the Tof distance data based on the Tof focus model and generate a focus weight corresponding to the scene image data; Identify the target focused object in the scene image data according to the focus weight, and determine the two-dimensional coordinate range of the target focused object in the scene image data; Determine the target Tof focus point position based on the Tof distance data and the two-dimensional coordinate range.
[0062] For example, in this embodiment, the Tof distance data is identified based on the Tof focus model, and the focus weight corresponding to the scene image data is generated. The target focus object in the scene image data is identified according to the focus weight, and the two-dimensional coordinate range of the target focus object in the scene image data is determined. When the user performs image acquisition based on the image acquisition device, the user will focus and click based on the scene objects in the environmental scene. In this embodiment, the Tof focus target object in the current scene image data is identified based on the Tof focus model. After determining the target object, the two-dimensional coordinate range of the target focus object in the scene image data is obtained. The center of the two-dimensional coordinate range is used as the target Tof focus point.
[0063] Through the above-mentioned method, in response to the image acquisition of the current environment scene by the camera device corresponding to the image acquisition device, the scene image data corresponding to the current environment scene is obtained, and the time-of-flight Tof sensing unit corresponding to the camera device transmits a preset beam to the current environment scene, and the Tof distance data corresponding to the scene image data is determined through the reflected beam corresponding to the object in the current environment scene, wherein the Tof distance data is used to indicate the visual distance between each object in the current environment scene and the camera device, and the first RGB pixel value distribution data corresponding to the scene image data is determined, and a multi-frame history image matching the first RGB pixel value distribution data is obtained from the image database corresponding to the image acquisition device. Image data, as well as multiple Tof historical distance data corresponding to multiple frames of historical image data, and multiple Tof focus points corresponding to multiple frames of historical image data, wherein each Tof focus point is a focus point selected by the user in the corresponding historical image. According to multiple Tof historical distance data and multiple Tof focus points, the user's shooting focus behavior based on the image acquisition device is learned, and the parameters of the preset AI model are updated based on the learning results to generate a Tof focus model, and the Tof distance data is identified based on the Tof focus model to generate the target Tof focus point corresponding to the scene image data, and the image acquisition device is automatically focused based on the target Tof focus point. In this way, the current shooting scene is automatically focused based on the user's shooting behavior, meeting the personalized focus shooting needs of different users.
[0064] Figure 2 is a block diagram of an AI user behavior learning device based on Tof focusing according to an exemplary embodiment. Figure 2 As shown, the device 100 is applied to an image acquisition device and includes: The acquisition module 110 is configured to acquire scene image data corresponding to the current environment scene in response to the image acquisition device corresponding to the camera device performing image acquisition on the current environment scene; The determination module 120 is configured to transmit a preset beam to the current environment scene based on a time-of-flight Tof sensing unit corresponding to the camera device, and determine Tof distance data corresponding to the scene image data through a reflected beam corresponding to an object in the current environment scene, wherein the Tof distance data is used to indicate a visual distance between each object in the current environment scene and the camera device; The acquisition module 130 is configured to determine the first RGB pixel value distribution data corresponding to the scene image data, and acquire multiple frames of historical image data matching the first RGB pixel value distribution data from an image database corresponding to the image acquisition device, as well as multiple Tof historical distance data corresponding to the multiple frames of historical image data, and multiple Tof focus points corresponding to the multiple frames of historical image data, wherein each Tof focus point is a focus point selected by a user in the corresponding historical image; The generating module 140 is configured to learn the user's shooting focusing behavior based on the image acquisition device according to the multiple Tof historical distance data and the multiple Tof focus points, and update the parameters of the preset AI model based on the learning result to generate a Tof focus model; The execution module 150 is configured to identify the Tof distance data based on the Tof focus model, generate a target Tof focus point position corresponding to the scene image data, and automatically focus the image acquisition device based on the target Tof focus point position.
[0065] Optionally, the acquisition module 130 includes: A first determining submodule is configured to determine a plurality of second RGB pixel value distribution data corresponding one to one to a plurality of frames of stored image data in an image database; A first generating submodule is configured to compare the scene image data with the multiple frames of stored image data frame by frame and pixel by pixel based on the multiple second RGB pixel value distribution data and the first RGB pixel value distribution data to generate multiple similarity indexes; The acquisition submodule is configured to acquire, based on multiple similarity indexes, multiple frames of stored images having the largest similarity index with the scene image data from the multiple frames of historical image data as the multiple frames of historical image data.
[0066] Optionally, the first generation submodule is configured as follows: Determine a first brightness comparison value, a first contrast comparison value, and a first structure comparison value between the scene image data and the first stored image data according to the third RGB pixel value distribution data and the first RGB pixel value distribution data, wherein the first stored image data is any multi-frame stored image data, and the third RGB pixel value distribution data is the RGB pixel value corresponding to the first stored image data; According to the first brightness comparison value, the first contrast comparison value and the first structure comparison value, a first similarity index between the first stored image data and the scene image data is determined based on the following formula:
[0067] in, is the X-axis component of the first brightness comparison value, is the Y-axis component of the first brightness comparison value, is the X-axis component of the first contrast comparison value, is the Y-axis component of the first contrast comparison value, is the first structure comparison value, is the first constant, is the second constant.
[0068] Optionally, the first generation submodule is configured as follows: Determining the size of a local window for image segmentation according to the image resolution of the multiple frames of stored image data, wherein the multiple local windows segment any stored image into multiple local images of the same size; Segmenting the second stored image data according to the multiple local windows to generate multiple frames of local image data, wherein the second stored image data is any multiple frames of stored image data; Determining a plurality of sub-similarity indexes according to third RGB pixel value distribution data and the first RGB pixel value distribution data, the third RGB pixel value distribution data being pixel values corresponding to the second stored image data; averaging the plurality of sub-similarity indexes to generate a first similarity index between the second stored image data and the scene image data; Based on the above steps, multiple frames of stored image data are traversed to generate multiple similarity indexes.
[0069] Optionally, the generating module 140 includes: The second generating submodule is configured to analyze the user's shooting focusing behavior based on a preset AI model according to a plurality of Tof historical distance data and a corresponding plurality of Tof focus points, and generate the user's focusing behavior data; A second determination submodule is configured to determine user personality preference data based on the multiple frames of historical image data and the focusing behavior data, where the user personality preference data is used to indicate the user's preference for shooting and focusing behavior; The third generating submodule is configured to extract features from the user's personality preference data through a preset feature algorithm to generate user feature preference data of the user; The fourth generation submodule is configured to update the parameters of the preset AI model based on the user feature preference data to generate a Tof focus model.
[0070] Optionally, the third generation submodule is configured as follows: Identify physical features in the first historical image data to generate item category parameters of the first historical image data, where the first historical image data is any multiple frames of historical image data; Determine the target item category corresponding to the first historical image data according to the position of the item category parameter in the first historical image data and the first Tof focus point position corresponding to the first historical image data; Generate a preference feature vector corresponding to the first historical image data through a preset feature algorithm according to the item category parameter and the target item category; Generating first user characteristic preference data according to the preference characteristic vector; Traversing multiple frames of historical image data to generate multiple user feature preference data; A weighted average is performed on multiple user characteristic preference data to generate user characteristic preference data.
[0071] Optionally, the execution module 150 is configured to: Identify the Tof distance data based on the Tof focus model and generate a focus weight corresponding to the scene image data; Identify the target focused object in the scene image data according to the focus weight, and determine the two-dimensional coordinate range of the target focused object in the scene image data; Determine the target Tof focus point position based on the Tof distance data and the two-dimensional coordinate range.
[0072] Through the above-mentioned method, in response to the image acquisition of the current environment scene by the camera device corresponding to the image acquisition device, the scene image data corresponding to the current environment scene is obtained, and the time-of-flight Tof sensing unit corresponding to the camera device transmits a preset beam to the current environment scene, and the Tof distance data corresponding to the scene image data is determined through the reflected beam corresponding to the object in the current environment scene, wherein the Tof distance data is used to indicate the visual distance between each object in the current environment scene and the camera device, and the first RGB pixel value distribution data corresponding to the scene image data is determined, and a multi-frame history image matching the first RGB pixel value distribution data is obtained from the image database corresponding to the image acquisition device. Image data, as well as multiple Tof historical distance data corresponding to multiple frames of historical image data, and multiple Tof focus points corresponding to multiple frames of historical image data, wherein each Tof focus point is a focus point selected by the user in the corresponding historical image. According to multiple Tof historical distance data and multiple Tof focus points, the user's shooting focus behavior based on the image acquisition device is learned, and the parameters of the preset AI model are updated based on the learning results to generate a Tof focus model, and the Tof distance data is identified based on the Tof focus model to generate the target Tof focus point corresponding to the scene image data, and the image acquisition device is automatically focused based on the target Tof focus point. In this way, the current shooting scene is automatically focused based on the user's shooting behavior, meeting the personalized focus shooting needs of different users.
[0073] The present disclosure also provides a computer-readable storage medium on which a computer program is stored. When the program is executed by a processor, the steps of the method described in any one of the aforementioned embodiments are implemented.
[0074] The present disclosure also provides an electronic device, including: processor; a memory for storing processor-executable instructions; The processor is configured to execute executable instructions in the memory to implement the method described in any one of the aforementioned embodiments.
[0075] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the present disclosure. This application is intended to cover any variations, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure. The specification and examples are to be considered as exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.
[0076] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.
Claims
1. An AI user behavior learning method based on Tof focusing, characterized in that: Applied to an image acquisition device, the method comprises: In response to capturing an image of a current environment scene by a camera device corresponding to the image capture device, acquiring scene image data corresponding to the current environment scene; Based on the time-of-flight Tof sensing unit corresponding to the camera device, a preset beam is emitted to the current environment scene, and Tof distance data corresponding to the scene image data is determined through the reflected beam corresponding to the object in the current environment scene, wherein the Tof distance data is used to indicate the visual distance between each object in the current environment scene and the camera device; Determine the first RGB pixel value distribution data corresponding to the scene image data, and obtain, from an image database corresponding to the image acquisition device, multiple frames of historical image data matching the first RGB pixel value distribution data, as well as multiple Tof historical distance data corresponding to the multiple frames of historical image data, and multiple Tof focus points corresponding to the multiple frames of historical image data, wherein each Tof focus point is a focus point selected by a user in the corresponding historical image; According to the multiple Tof historical distance data and the multiple Tof focus points, the user's shooting focus behavior based on the image acquisition device is learned, and the parameters of the preset AI model are updated based on the learning result to generate a Tof focus model; The Tof distance data is identified based on the Tof focusing model, a target Tof focusing point position corresponding to the scene image data is generated, and the image acquisition device is automatically focused based on the target Tof focusing point position.
2. The AI user behavior learning method based on Tof focusing according to claim 1 is characterized in that: The acquiring, from an image database corresponding to the image acquisition device, a plurality of frames of historical image data matching the first RGB pixel value distribution data comprises: Determine a plurality of second RGB pixel value distribution data corresponding one to one to a plurality of frames of stored image data in the image database; Based on the plurality of second RGB pixel value distribution data and the first RGB pixel value distribution data, comparing the scene image data with the plurality of frames of stored image data frame by frame and pixel by pixel to generate a plurality of similarity indexes; According to the multiple similarity indexes, a plurality of frames of stored images having the largest similarity index with the scene image data are obtained from the plurality of frames of historical image data as the plurality of frames of historical image data.
3. The AI user behavior learning method based on Tof focusing according to claim 2 is characterized in that: The step of comparing the scene image data with the multiple frames of stored image data frame by frame and pixel by pixel based on the multiple second RGB pixel value distribution data and the first RGB pixel value distribution data to generate multiple similarity indexes includes: Determining a first brightness comparison value, a first contrast comparison value, and a first structure comparison value between the scene image data and the first stored image data according to the third RGB pixel value distribution data and the first RGB pixel value distribution data, wherein the first stored image data is any one of the multiple frames of stored image data, and the third RGB pixel value distribution data is the RGB pixel value corresponding to the first stored image data; According to the first brightness comparison value, the first contrast comparison value, and the first structure comparison value, a first similarity index between the first stored image data and the scene image data is determined based on the following formula: Among them, the is the X-axis component of the first brightness comparison value, is the Y-axis component of the first brightness comparison value, is the X-axis component of the first contrast comparison value, is the Y-axis component of the first contrast comparison value, is the first structure comparison value, the is the first constant, is the second constant.
4. The AI user behavior learning method based on Tof focusing according to claim 2 is characterized in that: The step of comparing the scene image data with the multiple frames of stored image data frame by frame and pixel by pixel based on the multiple second RGB pixel value distribution data and the first RGB pixel value distribution data to generate multiple similarity indexes includes: Determining a local window size for image segmentation according to the image resolution of the multiple frames of stored image data, wherein the multiple local windows segment any stored image into multiple local images of the same size; Segmenting the second stored image data according to the multiple local windows to generate multiple frames of local image data, wherein the second stored image data is any of the multiple frames of stored image data; determining a plurality of sub-similarity indexes according to third RGB pixel value distribution data and the first RGB pixel value distribution data, wherein the third RGB pixel value distribution data is a pixel value corresponding to the second stored image data; Averaging the plurality of sub-similarity indexes to generate a first similarity index between the second stored image data and the scene image data; Based on the above steps, the plurality of frames of stored image data are traversed to generate the plurality of similarity indexes.
5. The AI user behavior learning method based on Tof focusing according to any one of claims 1-4, characterized in that: The method of learning the user's shooting focus behavior based on the image acquisition device according to the multiple Tof historical distance data and the multiple Tof focus points, and updating the parameters of the preset AI model based on the learning result to generate a Tof focus model includes: Based on the preset AI model, according to the multiple Tof historical distance data and the corresponding multiple Tof focus point positions, the user's shooting focus behavior analysis is performed to generate the user's focus behavior data; Determining the user personality preference data according to the multiple frames of historical image data and the focusing behavior data, wherein the user personality preference data is used to indicate the user's preference for shooting focusing behavior; Extracting features from the user's personality preference data using a preset feature algorithm to generate user feature preference data for the user; The parameters of the preset AI model are updated based on the user feature preference data to generate the Tof focusing model.
6. The AI user behavior learning method based on Tof focusing according to claim 5 is characterized in that: The extracting features of the user's personality preference data by using a preset feature algorithm to generate user feature preference data of the user includes: Identify physical features in first historical image data to generate item category parameters of the first historical image data, where the first historical image data is any one of the multiple frames of historical image data; Determine the target item category corresponding to the first historical image data according to the position of the item category parameter in the first historical image data and the first Tof focus point position corresponding to the first historical image data; According to the item category parameter and the target item category, generating a preference feature vector corresponding to the first historical image data by using the preset feature algorithm; generating the first user characteristic preference data according to the preference characteristic vector; Traversing the multiple frames of historical image data to generate multiple user feature preference data; The plurality of user characteristic preference data are weighted averaged to generate the user characteristic preference data.
7. The AI user behavior learning method based on Tof focusing according to claim 1 is characterized in that: The identifying the Tof distance data based on the Tof focusing model to generate a target Tof focus point position corresponding to the scene image data includes: Identify the Tof distance data based on the Tof focus model, and generate a focus weight corresponding to the scene image data; Identify the target focused object in the scene image data according to the focus weight, and determine the two-dimensional coordinate range of the target focused object in the scene image data; The target Tof focus point position is determined according to the Tof distance data and the two-dimensional coordinate range.
8. An AI user behavior learning device based on Tof focusing, characterized in that: Applied to an image acquisition device, the device comprises: A collection module, configured to acquire scene image data corresponding to the current environment scene in response to image collection of the current environment scene by a camera device corresponding to the image collection device; A determination module is configured to transmit a preset beam to the current environment scene based on a time-of-flight Tof sensing unit corresponding to the camera device, and determine Tof distance data corresponding to the scene image data through a reflected beam corresponding to an object in the current environment scene, wherein the Tof distance data is used to indicate a visual distance between each object in the current environment scene and the camera device; an acquisition module, configured to determine the first RGB pixel value distribution data corresponding to the scene image data, and acquire, from an image database corresponding to the image acquisition device, a plurality of frames of historical image data matching the first RGB pixel value distribution data, and a plurality of Tof historical distance data corresponding one-to-one to the plurality of frames of historical image data, and a plurality of Tof focus points corresponding one-to-one to the plurality of frames of historical image data, wherein each Tof focus point position is a focus point position selected by a user in the corresponding historical image; A generation module is configured to learn the user's shooting focus behavior based on the image acquisition device according to the multiple Tof historical distance data and the multiple Tof focus point positions, and update the parameters of the preset AI model based on the learning result to generate a Tof focus model; An execution module is configured to identify the Tof distance data based on the Tof focus model, generate a target Tof focus point position corresponding to the scene image data, and automatically focus the image acquisition device based on the target Tof focus point position.
9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method described in any one of claims 1 to 7 are implemented.
10. An electronic device, characterized in that: include: processor; a memory for storing processor-executable instructions; The processor is configured to execute executable instructions in the memory to implement the method described in any one of claims 1-7.
Citation Information
Patent Citations
Focusing method and device, storage medium and electronic equipment
CN109963072A
Focusing method and device, electronic device and computer readable storage medium
CN110149482A
Image focusing method and device and image acquisition equipment
CN112333379A
Airborne camera rapid focusing method
CN115037880A
System and method for image auto-focus
CN117082342A
Cited By
Three-dimensional scanning system for medical field
CN119359925A
3D scanning systems for medical applications
CN119359925B
