Content search method, device, electronic device and computer-readable storage medium
Through the image recognition technology of smart TVs, users' body movements are recognized and preset action sets are matched, and related content is recommended, which solves the problem of low content search efficiency in smart TVs and realizes efficient search without keyboard and voice input.
Patent Information
- Application Number
- CN202111004778.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-08-30
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2041-08-30
AI Technical Summary
In smart TVs, when searching content through remote control or voice input, users need to find a large number of video programs, resulting in inefficient search. Especially for people with unstandard Mandarin, keyboard and voice input do not support dialect input, which further reduces the Search efficiency.
By obtaining search images, image content recognition, search actions are determined, and matching them with the preset action set, and associated content is recommended to realize content search without keyboard and voice input.
Improves the efficiency of content search, simplifies user operations, is suitable for users in different dialects, and reduces search time.
Smart Images

Figure CN113743282B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of communication technology, and in particular to a content search method, device, electronic device and computer-readable storage medium. Background Art
[0002] With the development of smart TV related technologies, users can select the TV programs they want to watch from the smart TV through the remote control, or search for content by voice. However, when selecting through the remote control, it is necessary to find the content they want to watch among the numerous video programs classified according to the content, and the search process is generally time-consuming. When searching by inputting the name of the program they want to watch through the remote control or voice input, keyboard input and voice input usually do not support dialect input, which causes inconvenience to people who do not speak standard Mandarin, resulting in low search efficiency. Summary of the invention
[0003] Embodiments of the present application provide a content search method, device, electronic device, and computer-readable storage medium, which can improve content search efficiency.
[0004] A content search method provided in an embodiment of the present application includes:
[0005] In response to the content search instruction, obtaining a current search image;
[0006] Performing image content recognition on the current search image to determine the search action included in the current search image;
[0007] Matching the search action with a preset action set, determining a target action in the preset action set that matches the search action, wherein each action in the preset action set corresponds to at least one associated content;
[0008] The associated content of the target action is regarded as the content searched based on the search action and recommended.
[0009] Accordingly, an embodiment of the present application further provides a content search device, including:
[0010] An acquisition unit, configured to acquire a current search image in response to a content search instruction;
[0011] A first determining unit, configured to perform image content recognition on the current search image to determine a search action included in the current search image;
[0012] A second determination unit is used to match the search action with a preset action set, and determine a target action in the preset action set that matches the search action, wherein each action in the preset action set corresponds to at least one associated content;
[0013] The recommendation unit is used to take the associated content of the target action as the content searched based on the search action and recommend it.
[0014] Correspondingly, an embodiment of the present application also provides an electronic device, including a memory and a processor; the memory stores a computer program, and the processor is used to run the computer program in the memory to execute any content search method provided in the embodiment of the present application.
[0015] Correspondingly, an embodiment of the present application further provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store a computer program, and the computer program is loaded by a processor to execute any content search method provided in the embodiment of the present application.
[0016] The embodiment of the present application obtains the current search image in response to a content search instruction; performs image content recognition on the current search image to determine the search action contained in the current search image; matches the search action with a preset action set to determine a target action in the preset action set that matches the search action, wherein each action in the preset action set corresponds to at least one associated content; and uses the associated content of the target action as the content searched based on the search action and recommends it. This solution obtains the search image to perform image content recognition, determines the corresponding content based on the search action contained in the search image, and recommends the content. Content search can be achieved without the user having to use the keyboard or voice input, thereby improving search efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.
[0018] Figure 1 is a flow chart of a content search method provided by an embodiment of the present application;
[0019] Figure 2 is a schematic diagram of a content search device provided in an embodiment of the present application;
[0020] Figure 3 It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION
[0021] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.
[0022] The embodiments of the present application provide a content search method, device, electronic device and computer-readable storage medium. The content search device can be integrated in an electronic device, and the electronic device can be a terminal or other device.
[0023] The terminal may include a smart TV, a mobile phone, a wearable smart device, a tablet computer, a laptop computer, a personal computer (PC), and a vehicle-mounted computer.
[0024] It should be noted that the description order of the following embodiments is not intended to limit the preferred order of the embodiments.
[0025] This embodiment will be described from the perspective of a content search device, which is specifically integrated into a smart TV. Figure 1 As shown, the process of the content search method is as follows:
[0026] 101. In response to a content search instruction, obtain a current search image.
[0027] Among them, the content search instruction can be an instruction used to instruct the smart TV to obtain the current search image and perform subsequent content search and other operations. The content search instruction can be triggered by the user through a button on the remote control, or by clicking the designated control of the user interface displayed by the smart TV through the remote control (or finger), or by voice input.
[0028] The current search image may be an image used to search for content, and the current search image may be one image or include multiple images. The dynamic actions performed by the user may be identified through the multiple images included in the current search image.
[0029] For example, the user may press a designated button on the remote control of the smart TV to trigger a content search command of the smart TV. The smart TV responds to the content search command and obtains a current search image. The current search image may be sent to the smart TV by the user through a terminal such as a mobile phone connected to the smart TV, or obtained by the smart TV from its own memory.
[0030] Optionally, the current search image may also be acquired through a camera, that is, the step of “acquiring the current search image in response to the content search instruction” may specifically include:
[0031] In response to the content search instruction, displaying a content search interface;
[0032] Acquire at least one search image through an image acquisition device, and display the search image acquired by the image acquisition device on the content search interface;
[0033] A current search image is determined from the search images.
[0034] The content search interface may be a user interface (UI) displayed by a smart TV for interacting with a user.
[0035] The image acquisition device may be a device with an image acquisition function, such as a camera, a camcorder, or a still camera.
[0036] For example, the smart TV may respond to a content search instruction, display a content search interface, and start the camera to capture images. The smart TV may control the camera to obtain search images at a regular interval, for example, taking a picture at regular time intervals. The smart TV may also obtain search images in real time, such as videos, and display the search images captured by the camera on the content search interface. Displaying the search images allows users to confirm whether they are within the camera's capture range, and they may confirm whether their actions are correct through the search images.
[0037] The current search image is determined from at least one search image captured by the camera. If the camera only captures one search image, the search image is determined as the current search image. If the camera captures multiple search images, the user may not be ready for action when the camera is started. To avoid incorrectly recognized actions due to the user not being ready for action, the search image captured after the camera is started for a certain period of time can be used as the current search image.
[0038] Optionally, the search image may be preprocessed to filter out unqualified search images and reduce the number of images to be processed. For example, face detection may be performed on the search image to determine whether there is face information in the search image. If there is no face information, the search image captured by the camera does not contain the user, so the search image must not contain the search action, that is, the step of "determining the current search image from the search image" may specifically include:
[0039] Perform face detection on the search image;
[0040] If the search image contains face information, the search image is used as a candidate image;
[0041] A current search image is determined from the candidate images.
[0042] For example, face detection can be performed on the search image to determine whether there is face information in the search image. If there is face information in the search image, it means that there is a human body in the search image, and the search action recognition can be performed on the human body in the image, and the search image containing face information is used as a candidate image. The image with the earliest shooting time among the candidate images is used as the current search image, or the candidate image captured after the camera is started for a certain period of time is used as the current search image.
[0043] Optionally, a user who uses the search method of a smart TV for the first time to search may not be clear about the specific operation, and the user may be prompted to take action on the content search interface to perform the specific operation, that is, after the step of "responding to the content search instruction, displaying the content search interface", it may specifically include:
[0044] Obtaining action prompt information, the action prompt information includes any one of a prompt picture and a prompt text;
[0045] Display action prompt information on the content search interface.
[0046] The prompt information may be in the form of pictures or texts, etc., to prompt the user to perform body movements to search, for example, try to perform an archery action; or display a picture containing body movements.
[0047] For example, a user may trigger a content search command by pressing a designated button on a remote control. The smart TV may respond to the content search command by entering an action search mode, displaying a content search interface, and displaying a prompt picture including body movements on the content search interface. The smart TV may also be started to capture a search image and determine a current search image from the search images.
[0048] 102. Perform image content recognition on the current search image to determine the search action included in the current search image.
[0049] The search action may be a body movement used to search for content.
[0050] For example, the smart TV may extract edge features of the current search image based on an image edge detection algorithm (such as a Canny algorithm, a Sobel algorithm, or a Prewitt algorithm), and determine the extracted edge features as the search action included in the current search image.
[0051] Optionally, the reference name of the action may be determined by the extracted edge features, for example, an archery action, a jumping action, or an eye-covering action.
[0052] Optionally, the current search image may be identified by an action recognition model to obtain the search action of the current search image, that is, the step of "performing image content recognition on the current search image to determine the search action contained in the current search image" may specifically include:
[0053] Extract key points of the current search image through the action recognition model to obtain the key points of the current search image;
[0054] Determine the search action contained in the current search image based on the key points.
[0055] Among them, the key points may be the key points of human body movements, and the human skeleton information may be determined based on the key points to further determine the movements made by the human body. For example, the key points may be the positions of the joints of the human body.
[0056] For example, the key points of the current search image can be detected by an action recognition model based on a bottom-up posture estimation method (for example, openpose algorithm) or a bottom-up posture estimation method (for example, Alphapose algorithm), and the key points of the human body in the current search image can be obtained. The search action can be obtained based on the key points, for example, by connecting lines based on the key points to obtain the search action contained in the current search image.
[0057] In one embodiment, after the action recognition model passes the key point detection, the reference name of the search action of the current search image can be determined based on the key point detection, such as an archery action, a jumping action, or a circling action.
[0058] 103. Match the search action with a preset action set to determine a target action in the preset action set that matches the search action, wherein each action in the preset action set corresponds to at least one associated content.
[0059] Among them, the preset action set can be a pre-set action set, and the preset action set can include relevant action information corresponding to multiple actions. For example, the preset action set can include multiple pictures containing actions, or the preset action set can include feature vectors or feature information representing actions, etc. Each action corresponds to at least one associated content, and the associated content can be content corresponding to the action, such as video, music, and articles, etc.
[0060] For example, the smart TV may specifically match the search action with relevant action information corresponding to each action in the preset set, calculate the similarity, and determine the action with the highest similarity in the preset action set as the target action matched by the search action.
[0061] Optionally, if the actions contained in the preset action set are reference names for each action, image content recognition can be performed on the current search image to determine the reference names of the actions contained in the current search image, and the target action in the preset action set that matches the search action can be determined based on the reference names.
[0062] Optionally, during a content search, if no valid action (action in the preset set) is identified in the current search image being identified, other search images may be obtained for image content identification to increase the probability of successful content search, i.e., the step of "matching the search action with the preset action set to determine the target action in the preset action set that matches the search action" may specifically be:
[0063] If there is an action matching the search action in the preset action set, the matching action is used as the target action corresponding to the search action;
[0064] If there is no action matching the search action in the preset action set, the target search image is determined from the search image based on the preset rules, and the target search image is used as the current search image. The target search image is returned to perform image content recognition on the target search image to determine the search action contained in the target search image.
[0065] For example, if there is an action in the preset action set that matches the search action, the action in the preset action set will be used as the target action corresponding to the search action; if there is no action in the preset action set that matches the action included in the current search image, the smart TV will determine a search image from the collected search images as the current search image, for example, the next search image in the time sequence of the current search image is used as the current search image, or a search image whose time interval with the current search image exceeds a threshold is used as the current search image, and image content recognition is performed again.
[0066] Optionally, in order to prevent the smart TV from continuously executing the recognition action process, a loop end condition can be set. For example, if the smart TV cannot match any action in the preset action set after performing image content recognition for 10 consecutive times, the current search content will be terminated. Optionally, the smart TV can also display a prompt message to prompt the user to perform the action again for recognition.
[0067] 104. The related content of the target action is used as the content searched based on the search action and recommended.
[0068] For example, the smart TV may specifically use the associated content corresponding to the target action as the content searched according to the search action, and display the content to recommend it to the user.
[0069] Optionally, in order to reduce the content occupation of the smart TV, each action in the preset action set may correspond to identification information of the associated content, that is, the step of "using the associated content of the target action as the content searched based on the search action and recommending it" may specifically include:
[0070] Mapping the target action to obtain identification information of the associated content corresponding to the target action;
[0071] Acquire related content based on identification information;
[0072] Using the related content as the content searched based on the search action;
[0073] Display searched content for recommendation.
[0074] For example, the smart TV may map the target action, determine identification information of the associated content corresponding to the target action, obtain the corresponding associated content from the memory according to the identification information, and display it.
[0075] Optionally, in order to further reduce the memory usage of the smart TV, the smart TV can obtain corresponding related content from the server based on the identification information of the related content, use the related content as the content searched based on the search action, and display the content to complete the search process.
[0076] The embodiment of the present application obtains the current search image in response to a content search instruction; performs image content recognition on the current search image to determine the search action contained in the current search image; matches the search action with a preset action set to determine a target action in the preset action set that matches the search action, wherein each action in the preset action set corresponds to at least one associated content; and uses the associated content of the target action as the content searched based on the search action and recommends it. This solution obtains the search image to perform image content recognition, determines the corresponding content based on the search action contained in the search image, and recommends the content. Content search can be achieved without the user having to use the keyboard or voice input, thereby improving search efficiency.
[0077] In order to facilitate better implementation of the content search method provided in the embodiment of the present application, a content search device is also provided in one embodiment. The meanings of the terms are the same as those in the above-mentioned content search method, and the specific implementation details can refer to the description in the method embodiment.
[0078] The content search device can be integrated into an electronic device, such as Figure 2 As shown, the content search device may include: an acquisition unit 301, a first determination unit 302, a second determination unit 303, and a recommendation unit 304, which are specifically as follows:
[0079] Acquisition unit 301: used to acquire the current search image in response to a content search instruction.
[0080] For example, the user may press a designated button on the remote control of the smart TV to trigger a content search command of the smart TV. The smart TV responds to the content search command and obtains a current search image. The current search image may be sent to the smart TV by the user through a terminal such as a mobile phone connected to the smart TV, or obtained by the smart TV from its own memory.
[0081] Optionally, the acquisition unit 301 may include a response subunit, an acquisition subunit, and a determination subunit, specifically:
[0082] Response subunit: used to respond to the content search instruction and display the content search interface;
[0083] An acquisition subunit: used to acquire at least one search image through an image acquisition device, and display the search image acquired by the image acquisition device on the content search interface;
[0084] Determine subunit: used to determine the current search image from the search images.
[0085] For example, the smart TV may respond to a content search instruction, display a content search interface, and start the camera to capture images. The smart TV may control the camera to obtain search images at a regular interval, for example, taking a picture at regular time intervals. The smart TV may also obtain search images in real time, for example, video, and display the search images captured by the camera on the content search interface. Displaying the search images allows the user to confirm whether they are within the camera's capture range, and may confirm whether the action is correct through the search images.
[0086] The current search image is determined from at least one search image captured by the camera. If the camera only captures one search image, the search image is determined as the current search image. If the camera captures multiple search images, the user may not be ready for action when the camera is started. To avoid incorrectly recognized actions due to the user not being ready for action, the search image captured a certain time after the camera is started can be used as the current search image.
[0087] Optionally, the determination subunit may include a detection module, a judgment module and an image determination module, specifically:
[0088] Detection module: used to perform face detection on search images;
[0089] Judgment module: used to take the search image as a candidate image if the search image contains face information;
[0090] Image determination module: used to determine the current search image from the candidate images.
[0091] For example, face detection can be performed on the search image to determine whether there is face information in the search image. If there is face information in the search image, it means that there is a human body in the search image. Search action recognition can be performed on the human body in the search image, and the search image containing face information is used as a candidate image. The image with the earliest shooting time among the candidate images is used as the current search image, or the candidate image captured a certain period of time after the camera is started is used as the current search image.
[0092] Optionally, the acquisition unit 301 may include an acquisition subunit and a display subunit, specifically:
[0093] Acquisition subunit: used to acquire action prompt information, which includes any one of prompt pictures and prompt texts;
[0094] Display subunit: used to display action prompt information on the content search interface.
[0095] For example, a user may trigger a content search command by pressing a designated button on a remote control. The smart TV may respond to the content search command by entering an action search mode, displaying a content search interface, and displaying a prompt picture including body movements on the content search interface. The smart TV may also be started to capture a search image and determine a current search image from the search images.
[0096] The first determining unit 302 is used to perform image content recognition on the current search image to determine the search action included in the current search image.
[0097] For example, the smart TV may extract edge features of the current search image based on an image edge detection algorithm (such as a Canny algorithm, a Sobel algorithm, or a Prewitt algorithm), and determine the extracted edge features as the search action included in the current search image.
[0098] Optionally, the reference name of the action may be determined by the extracted edge features, for example, an archery action, a jumping action, or an eye-covering action.
[0099] Optionally, the first determining unit 302 may include an extracting subunit and a determining subunit, specifically:
[0100] Extraction subunit: used to extract key points of the current search image through the action recognition model to obtain the key points of the current search image;
[0101] Determine subunit: used to determine the search action contained in the current search image based on key points.
[0102] For example, the key points of the current search image can be detected by an action recognition model based on a bottom-up posture estimation method (for example, openpose algorithm) or a bottom-up posture estimation method (for example, Alphapose algorithm), and the key points of the human body in the current search image can be obtained. The search action can be obtained based on the key points, for example, by connecting lines based on the key points to obtain the search action contained in the current search image.
[0103] In one embodiment, after the action recognition model passes the key point detection, the reference name of the search action of the current search image can be determined based on the key point detection, for example, an archery action or a jumping action.
[0104] The second determining unit 303 is used to match the search action with the preset action set, and determine the target action in the preset action set that matches the search action, wherein each action in the preset action set corresponds to at least one associated content.
[0105] For example, the smart TV may specifically match the search action with relevant action information corresponding to each action in the preset set, calculate the similarity, and determine the action with the highest similarity in the preset action set as the target action matched by the search action.
[0106] Optionally, if the actions contained in the preset action set are reference names for each action, image content recognition can be performed on the current search image to determine the reference names of the actions contained in the current search image, and the target action in the preset action set that matches the search action can be determined based on the reference names.
[0107] Optionally, the second determining unit 303 may include a first circulation subunit and a second circulation subunit, specifically:
[0108] The first loop subunit is used for, if there is an action matching the search action in the preset action set, taking the matching action as the target action corresponding to the search action;
[0109] The second loop sub-unit is used to determine the target search image from the search image based on preset rules if there is no action matching the search action in the preset action set, and use the target search image as the current search image, and return to perform image content recognition on the target search image to determine the search action contained in the target search image.
[0110] For example, if there is an action in the preset action set that matches the search action, the action in the preset action set will be used as the target action corresponding to the search action; if there is no action in the preset action set that matches the action included in the current search image, the smart TV will determine a search image from the collected search images as the current search image, for example, the next search image in the time sequence of the current search image is used as the current search image, or a search image whose time interval with the current search image exceeds a threshold is used as the current search image, and image content recognition is performed again.
[0111] Optionally, in order to prevent the smart TV from continuously executing the recognition action process, a loop end condition can be set. For example, if the smart TV continuously recognizes the image content for 10 times and still cannot match any action in the preset action set, the current search content will be terminated. Optionally, the smart TV can also display a prompt message to prompt the user to perform the action again for recognition.
[0112] Recommendation unit 304: used to take the associated content of the target action as the content searched based on the search action and recommend it.
[0113] For example, the smart TV may specifically use the associated content corresponding to the target action as the content searched according to the search action, and display the content to recommend it to the user.
[0114] Optionally, the recommendation unit 304 may include a mapping subunit, an acquisition subunit, a content determination subunit, and a display subunit. Specifically:
[0115] Mapping subunit: used to map the target action to obtain identification information of the associated content corresponding to the target action;
[0116] Acquisition subunit: used for acquiring related content based on identification information;
[0117] A content determination subunit: used for taking the associated content as the content searched based on the search action;
[0118] Display subunit: used to display searched content for recommendation.
[0119] For example, the smart TV may map the target action, determine identification information of the associated content corresponding to the target action, obtain the corresponding associated content from the memory according to the identification information, and display it.
[0120] Optionally, in order to further reduce the memory usage of the smart TV, the smart TV can obtain corresponding related content from the server based on the identification information of the related content, use the related content as the content searched based on the search action, and display the content to complete the search process.
[0121] The content search device of the embodiment of the present application can obtain the current search image in response to the content search instruction through the acquisition unit 301; perform image content recognition on the current search image through the first determination unit 302 to determine the search action contained in the current search image; the second determination unit 303 matches the search action with the preset action set to determine the target action that matches the search action in the preset action set, and each action in the preset action set corresponds to at least one associated content; the recommendation unit 304 uses the associated content of the target action as the content searched based on the search action and recommends it. This solution obtains the search image to perform image content recognition, determines the corresponding content based on the search action contained in the search image, and recommends the content. The user does not need to use the keyboard or voice input to implement content search, thereby improving search efficiency.
[0122] The present application also provides an electronic device, which may be a terminal. Figure 3 As shown, it shows a schematic diagram of the structure of the electronic device involved in the embodiment of the present application, specifically:
[0123] The electronic device may include components such as a processor 1001 with one or more processing cores, a memory 1002 with one or more computer-readable storage media, a power supply 1003, and an input unit 1004. Those skilled in the art will appreciate that Figure 3 The electronic device structure shown in the figure does not constitute a limitation on the electronic device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently.
[0124] The processor 1001 is the control center of the electronic device. It uses various interfaces and lines to connect various parts of the entire electronic device. By running or executing software programs and / or modules stored in the memory 1002 and calling data stored in the memory 1002, it performs various functions of the electronic device and processes data, thereby monitoring the electronic device as a whole. Optionally, the processor 1001 may include one or more processing cores; preferably, the processor 1001 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, computer program, etc., and the modem processor mainly processes wireless communication. It is understandable that the above-mentioned modem processor may not be integrated into the processor 1001.
[0125] The memory 1002 can be used to store software programs and modules. The processor 1001 executes various functional applications and data processing by running the software programs and modules stored in the memory 1002. The memory 1002 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, a computer program required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the electronic device, etc. In addition, the memory 1002 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 1002 may also include a memory controller to provide the processor 1001 with access to the memory 1002.
[0126] The electronic device also includes a power supply 1003 for supplying power to each component. Preferably, the power supply 1003 can be logically connected to the processor 1001 through a power management system, so as to manage charging, discharging, and power consumption through the power management system. The power supply 1003 can also include one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, power status indicators, and other arbitrary components.
[0127] The electronic device may further include an input unit 1004, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.
[0128] Although not shown, the electronic device may further include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 1001 in the electronic device will load the executable files corresponding to the processes of one or more computer programs into the memory 1002 according to the following instructions, and the processor 1001 will run the computer programs stored in the memory 1002, thereby realizing various functions, as follows:
[0129] In response to the content search instruction, obtaining a current search image;
[0130] Perform image content recognition on the current search image to determine the search action contained in the current search image;
[0131] Matching the search action with a preset action set, determining a target action in the preset action set that matches the search action, each action in the preset action set corresponding to at least one associated content;
[0132] The related content of the target action is regarded as the content searched based on the search action and recommended.
[0133] The specific implementation of the above operations can be found in the previous embodiments and will not be described in detail here.
[0134] As can be seen from the above, the electronic device of the embodiment of the present application can obtain the current search image by responding to the content search instruction; perform image content recognition on the current search image to determine the search action contained in the current search image; match the search action with the preset action set to determine the target action in the preset action set that matches the search action, each action in the preset action set corresponds to at least one associated content; use the associated content of the target action as the content searched based on the search action, and recommend it. This solution performs image content recognition by obtaining the search image, determines the corresponding content based on the search action contained in the search image, and recommends the content. The user does not need to use the keyboard or voice input to implement content search, thereby improving search efficiency.
[0135] According to one aspect of the present application, a computer program product or a computer program is provided, the computer program product or the computer program including computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of an electronic device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the electronic device executes the methods provided in various optional implementations in the above-mentioned embodiments.
[0136] Those skilled in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by a computer program, or by controlling related hardware through a computer program. The computer program may be stored in a computer-readable storage medium and loaded and executed by a processor.
[0137] To this end, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. The computer program can be loaded by a processor to execute any content search method provided in the embodiment of the present application.
[0138] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.
[0139] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.
[0140] Since the computer program stored in the computer-readable storage medium can execute any content search method provided in the embodiments of the present application, the beneficial effects that can be achieved by any content search method provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.
[0141] The above is a detailed introduction to a content search method, device, electronic device and computer-readable storage medium provided in the embodiments of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.
Claims
1. A content search method, characterized in that: include: In response to the content search instruction, obtaining a current search image; Extract key points of the current search image through an action recognition model to obtain key points of the current search image, and connect the key points to determine the search action included in the current search image and the reference name of the search action; Determining a target action matching the search action in a preset action set according to the reference name of the search action, wherein the actions included in the preset action set are the reference names of each action, and each action corresponds to at least one associated content; The associated content of the target action is regarded as the content searched based on the search action and recommended.
2. The method according to claim 1, characterized in that The taking the associated content of the target action as the content searched based on the search action and recommending it includes: Mapping the target action to obtain identification information of the associated content corresponding to the target action; Acquire the associated content based on the identification information; Using the associated content as content searched based on the search action; The searched content is displayed for recommendation.
3. The method according to claim 1, characterized in that The step of obtaining the current search image in response to the content search instruction includes: In response to the content search instruction, displaying a content search interface; Acquire at least one search image through an image acquisition device, and display the search image acquired by the image acquisition device on the content search interface; A current search image is determined from the search images.
4. The method according to claim 3, characterized in that After displaying the content search interface in response to the content search instruction, the method further includes: Acquire action prompt information, where the action prompt information includes any one of a prompt picture and a prompt text; The action prompt information is displayed on the content search interface.
5. The method according to claim 3, characterized in that: The step of determining a target action in a preset action set that matches the search action according to the reference name of the search action includes: If there is an action matching the search action in the preset action set, taking the matching action as the target action corresponding to the search action; If there is no action matching the search action in the preset action set, a target search image is determined from the search image based on preset rules, and the target search image is used as the current search image. The process returns to perform image content recognition on the target search image to determine the search action contained in the target search image.
6. The method according to claim 3, characterized in that: The step of determining a current search image from the search images comprises: Performing face detection processing on the search image; If the search image contains face information, taking the search image as a candidate image; A current search image is determined from the candidate images.
7. A content search device, characterized in that: include: An acquisition unit, configured to acquire a current search image in response to a content search instruction; A first determining unit is used to extract key points of the current search image through an action recognition model to obtain key points of the current search image, and to connect the key points to determine the search action included in the current search image and the reference name of the search action; A second determining unit is used to determine a target action matching the search action in a preset action set according to the reference name of the search action, wherein the actions included in the preset action set are reference names of each action, and each action corresponds to at least one associated content; The recommendation unit is used to take the associated content of the target action as the content searched based on the search action and recommend it.
8. An electronic device, characterized in that: It comprises a memory and a processor; the memory stores a computer program, and the processor is used to run the computer program in the memory to execute the content search method according to any one of claims 1 to 6.
9. A computer-readable storage medium, characterized in that: The computer-readable storage medium is used to store a computer program, and the computer program is loaded by a processor to execute the content search method according to any one of claims 1 to 6.
