Video interaction method, device and equipment based on user requirements

Abstract

The invention provides a video interaction method, device and equipment based on user requirements, and relates to the technical field of video interaction. In the application, first, target interaction request information sent by a target user through a corresponding target terminal device is obtained, and the target interaction request information is analyzed to obtain target position information corresponding to the target user; secondly, based on the target position information, determining a target real-time monitoring video; and then, transmitting the target real-time monitoring video to target terminal equipment corresponding to the target user so as to complete real-time video interaction between the electronic equipment and the target terminal equipment. Based on the content, the problem that the reliability of video interaction is relatively low in the prior art can be improved.

Description

Technical Field

[0001] The present application relates to the field of video interaction technology, and in particular to a video interaction method, device and equipment based on user needs. Background Art

[0002] With the popularization of smart devices and the continuous advancement of technology, video interaction technology has been widely used in many fields, including telemedicine, video surveillance, online education, video conferencing, customer service, etc. Among them, video surveillance can be used as a basis for users to choose travel. For example, in the existing technology, users can search for videos of areas of interest to understand the situation in the corresponding area. However, the videos searched by users are generally based on those sent by other users. Therefore, they are generally not real-time, making the information obtained unreliable, that is, there is a problem of relatively low reliability of video interaction. Summary of the invention

[0003] In view of this, the purpose of the present application is to provide a video interaction method, device and equipment based on user needs, so as to improve the problem of relatively low reliability of video interaction existing in the prior art.

[0004] To achieve the above objectives, this application adopts the following technical solutions:

[0005] A video interaction method based on user needs is applied to an electronic device, wherein the video interaction method based on user needs comprises:

[0006] Acquire target interaction request information sent by a target user through a corresponding target terminal device, and parse the target interaction request information to obtain target location information corresponding to the target user, wherein the target location information is used to reflect the area of ​​interest of the target user;

[0007] Based on the target location information, determine a target real-time monitoring video, wherein the target real-time monitoring video is obtained by performing real-time video monitoring of the area of ​​interest by a video monitoring device deployed in the area of ​​interest reflected by the target location information;

[0008] The target real-time monitoring video is transmitted to the target terminal device corresponding to the target user to complete the real-time video interaction between the electronic device and the target terminal device.

[0009] In a preferred embodiment of the present application, in the above-mentioned video interaction method based on user needs, the step of determining the target real-time monitoring video based on the target location information includes:

[0010] Determine each video surveillance device deployed in the area of ​​interest reflected by the target location information to form at least one target video surveillance device;

[0011] Each of the target video monitoring devices is controlled to start and perform video monitoring, and a target real-time monitoring video corresponding to each of the target video monitoring devices is obtained.

[0012] In a preferred embodiment of the present application, in the above-mentioned video interaction method based on user needs, the step of determining each video surveillance device deployed in the area of ​​interest reflected by the target location information to form at least one target video surveillance device includes:

[0013] Determine each video surveillance device deployed in the area of ​​interest reflected by the target location information to form at least one candidate video surveillance device;

[0014] When the number of the at least one candidate video surveillance device is less than or equal to a predetermined reference number, taking the at least one candidate video surveillance device as a target event monitoring device;

[0015] When the number of the at least one candidate video surveillance device is greater than the reference number, at least one candidate video surveillance device is determined as a target event monitoring device from the at least one candidate video surveillance device based on a predetermined device screening rule.

[0016] In a preferred embodiment of the present application, in the above-mentioned video interaction method based on user needs, when the number of devices of the at least one candidate video surveillance device is greater than the reference number, based on a predetermined device screening rule, the step of determining at least one candidate video surveillance device from the at least one candidate video surveillance device as a target event monitoring device includes:

[0017] When the number of the at least one candidate video surveillance device is greater than the reference number, obtaining region description information of the local surveillance area corresponding to each of the candidate video surveillance devices, wherein the region description information includes video and / or text;

[0018] Acquire target user demand information sent by the target user through the target terminal device, wherein the target user demand information is used to reflect the content of interest to the target user;

[0019] For each of the candidate video surveillance devices, determining a degree of match between the area description information of the local surveillance area corresponding to the candidate video surveillance device and the target user demand information;

[0020] Each candidate video surveillance device with a matching degree greater than or equal to a predetermined reference matching degree is taken as a target event monitoring device, or a target number of candidate video surveillance devices with the largest matching degrees are taken as target event monitoring devices.

[0021] In a preferred embodiment of the present application, in the above-mentioned video interaction method based on user needs, the step of determining the target real-time monitoring video based on the target location information includes:

[0022] Determine the video surveillance equipment deployed in the area of ​​interest reflected by the target location information to form a target video surveillance equipment, wherein the video surveillance equipment is an airborne equipment of a drone;

[0023] The target video monitoring device is controlled to move in the area of ​​interest reflected by the target position information, and video monitoring is performed during the movement to obtain a corresponding target real-time monitoring video.

[0024] In a preferred embodiment of the present application, in the above-mentioned video interaction method based on user needs, the step of controlling the target video monitoring device to move in the area of ​​interest reflected by the target position information, and performing video monitoring during the movement to obtain the corresponding target real-time monitoring video includes:

[0025] Acquire target user demand information sent by the target user through the target terminal device, wherein the target user demand information is used to reflect the content of interest to the target user;

[0026] Based on the target user demand information, determining a target motion path in the region of interest reflected by the target position information, wherein the target motion path includes at least a starting point and an end point;

[0027] The target video monitoring device is controlled to move along the target motion path in the area of ​​interest reflected by the target position information, and video monitoring is performed through the target video monitoring device during the movement to obtain a corresponding target real-time monitoring video.

[0028] In a preferred embodiment of the present application, in the above-mentioned video interaction method based on user demand, the step of determining the target motion path in the region of interest reflected by the target position information based on the target user demand information includes:

[0029] Determine each historical user having demand correlation with the target user based on the target user demand information to form at least one related historical user, wherein the historical user refers to a user who has historically performed real-time video interaction with the electronic device;

[0030] Acquire historical real-time monitoring videos formed in the history of each of the relevant historical users to form at least one historical real-time monitoring video;

[0031] Performing multimodal feature mining and aggregation on the at least one historical real-time monitoring video and the target user demand information to form a multimodal user demand feature corresponding to the target user, wherein the multimodal user demand feature is used to reflect the semantic information possessed by the at least one historical real-time monitoring video and the target user demand information;

[0032] Performing path planning processing on the area of ​​interest reflected by the target position information to form a plurality of candidate motion paths, and, for each of the candidate motion paths, respectively obtaining node description information of each path node in the candidate motion path, and combining the node description information of each path node to form path description information corresponding to the candidate motion path, wherein the node description information includes video and / or text;

[0033] Performing feature mining on the path description information corresponding to each of the candidate motion paths to form a path description feature corresponding to each of the candidate motion paths;

[0034] The similarity between the path description feature corresponding to each of the candidate motion paths and the multimodal user demand feature is determined respectively, and the candidate motion path with the maximum similarity is determined as the target motion path.

[0035] In a preferred embodiment of the present application, in the above-mentioned video interaction method based on user needs, the step of determining each historical user having demand correlation with the target user based on the target user demand information to form at least one related historical user includes:

[0036] Determine a historical user relationship network constructed based on each historical user and the historical user demand information corresponding to each historical user, and, based on the target user and the target user demand information, perform a first update on the historical user relationship network to form a first updated user relationship network, wherein, during the first update process, connect each historical user corresponding to each historical user demand information whose correlation with the target user demand information is greater than or equal to a preset correlation with the target user demand information to indicate that the historical user has demand correlation with the target user;

[0037] Perform feature mining on each of the to-be-processed user demand information respectively, and output the to-be-processed user demand features corresponding to each of the to-be-processed user demand information, wherein each of the to-be-processed user demand information belongs to the historical user demand information or the target user demand information;

[0038] For each user to be processed in the first updated user relationship network, a local user relationship network centered on the user to be processed is determined in the first updated user relationship network, wherein each of the users to be processed belongs to the historical user or the target user;

[0039] Performing random walks in the first updated user relationship network to form a plurality of random walk paths, and randomly serializing the plurality of random walk paths to form a corresponding target path sequence, wherein the path end point of each of the random walk paths is the target user;

[0040] According to the order relationship of the multiple random walk paths in the target path sequence, and according to the direction from the path starting point to the path ending point in the random walk path, the currently traversed user to be processed is determined, and based on the current user demand features of the currently traversed user to be processed corresponding to each user to be processed in the local user relationship network of the currently traversed user to be processed, the current user demand features of the currently traversed user to be processed corresponding to the current user demand features to be processed are associated with mining to obtain the corresponding associated user demand features, and the associated user demand features are used as new current user demand features to be processed, and after the current user demand features to be processed corresponding to the target user are associated with mining in the last random walk path, based on the feature similarity between the current user demand features to be processed corresponding to each user to be processed, the first updated user relationship network is second updated to form a second updated user relationship network;

[0041] In the second updated user relationship network, each historical user having a connection relationship with the target user is determined to form at least one related historical user.

[0042] The present application also provides a video interaction device based on user needs, which is applied to an electronic device, wherein the video interaction device based on user needs includes:

[0043] a location information determination module, configured to obtain target interaction request information sent by a target user through a corresponding target terminal device, and parse the target interaction request information to obtain target location information corresponding to the target user, wherein the target location information is used to reflect an area of ​​interest of the target user;

[0044] A monitoring video determination module, used to determine a target real-time monitoring video based on the target location information, wherein the target real-time monitoring video is obtained by performing real-time video monitoring of the area of ​​interest by a video monitoring device deployed in the area of ​​interest reflected by the target location information;

[0045] The monitoring video transmission module is used to transmit the target real-time monitoring video to the target terminal device corresponding to the target user to complete the real-time video interaction between the electronic device and the target terminal device.

[0046] Based on the above, the present application also provides an electronic device, including:

[0047] Memory for storing computer programs;

[0048] The processor connected to the memory is used to execute the computer program stored in the memory to implement the above-mentioned video interaction method based on user needs.

[0049] The video interaction method, device and equipment based on user needs provided by the present application first obtain the target interaction request information sent by the target user through the corresponding target terminal device, and parse the target interaction request information to obtain the target location information corresponding to the target user; secondly, based on the target location information, determine the target real-time monitoring video; then, transmit the target real-time monitoring video to the target terminal device corresponding to the target user to complete the real-time video interaction between the electronic device and the target terminal device. Based on the above content, after analyzing the target location information of the area of ​​interest of the target user, the video monitoring device deployed in the area of ​​interest reflected by the target location information can be used to perform real-time video monitoring of the area of ​​interest, so that the obtained target real-time monitoring video has high real-time performance, thereby ensuring that the obtained relevant information has high reliability, and therefore, the problem of relatively low reliability of video interaction existing in the prior art can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings.

[0051] Figure 1 A structural block diagram of an electronic device provided in an embodiment of the present application.

[0052] Figure 2 A flowchart of a video interaction method based on user needs provided in an embodiment of the present application.

[0053] Figure 3 A schematic diagram of a first updated user relationship network provided in an embodiment of the present application.

[0054] Figure 4 A schematic diagram of a local user relationship network provided in an embodiment of the present application.

[0055] Figure 5 A block diagram of a video interaction device based on user needs provided in an embodiment of the present application. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. The components of the embodiments of the present application described and shown in the drawings here can be arranged and designed in various different configurations.

[0057] Therefore, the following detailed description of the embodiments of the present application provided in the accompanying drawings is not intended to limit the scope of the present application for which protection is sought, but merely represents selected embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0058] like Figure 1 As shown, an embodiment of the present application provides an electronic device, wherein the electronic device may include a memory, a processor, and a video interaction device based on user needs.

[0059] In detail, the memory and the processor are electrically connected directly or indirectly to achieve data transmission or interaction. For example, the memory and the processor can be electrically connected through one or more communication buses or signal lines. The video interaction device based on user needs includes at least one software function module stored in the memory in the form of software or firmware. The processor is used to execute an executable computer program stored in the memory, for example, the software function module and computer program included in the video interaction device based on user needs, so as to implement the video interaction method based on user needs provided in an embodiment of the present application.

[0060] Optionally, the memory can be, but is not limited to, random access memory (RAM), read only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable read-only memory (Electric Erasable Programmable Read-Only Memory, EEPROM), etc.

[0061] Furthermore, the processor may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), a system on chip (SoC), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0062] Understandably, Figure 1 The structure shown is for illustration only, and the electronic device may also include Figure 1 More or fewer components as shown, or with Figure 1 The different configurations shown, for example, may also include a communication unit for exchanging information with other devices (such as drone devices, etc.).

[0063] Combination Figure 2 The embodiment of the present application also provides a video interaction method based on user needs that can be applied to the above electronic device. Among them, the method steps defined in the process related to the video interaction method based on user needs can be implemented by the electronic device.

[0064] The following will Figure 2 The specific process shown is explained in detail.

[0065] Step S110: acquiring target interaction request information sent by a target user through a corresponding target terminal device, and parsing the target interaction request information to obtain target location information corresponding to the target user.

[0066] In an embodiment of the present application, the electronic device can obtain the target interaction request information sent by the target user through the corresponding target terminal device, and parse the target interaction request information to obtain the target location information corresponding to the target user. The target location information is used to reflect the area of ​​interest of the target user, such as a scenic spot searched by the target user.

[0067] Step S120: determining a target real-time monitoring video based on the target location information.

[0068] In an embodiment of the present application, after obtaining the target location information, the electronic device can determine the target real-time monitoring video based on the target location information. The target real-time monitoring video is obtained by performing real-time video monitoring of the area of ​​interest by a video monitoring device deployed in the area of ​​interest reflected by the target location information. In other words, the electronic device can control the corresponding video monitoring device to perform video monitoring to obtain the corresponding target real-time monitoring video, and then the target real-time monitoring video can be stored in the corresponding database.

[0069] Step S130, transmitting the target real-time monitoring video to the target terminal device corresponding to the target user, so as to complete the real-time video interaction between the electronic device and the target terminal device.

[0070] In an embodiment of the present application, after determining the target real-time monitoring video, the electronic device can transmit the target real-time monitoring video to the target terminal device corresponding to the target user to complete the real-time video interaction between the electronic device and the target terminal device. Specifically, the target real-time monitoring video can be stored in the electronic device, so that the target real-time monitoring video can be directly transmitted to the target terminal device. Alternatively, the target real-time monitoring video can also be stored in other devices, so that the electronic device can send the storage address of the target real-time monitoring video to the target terminal device, so that the target terminal device can access the target real-time monitoring video based on the storage address to obtain it.

[0071] Based on the above content, after analyzing the target location information of the area of ​​interest of the target user, real-time video monitoring of the area of ​​interest can be performed by deploying video surveillance equipment in the area of ​​interest reflected by the target location information, so that the acquired target real-time monitoring video has high real-time performance, thereby ensuring that the acquired relevant information has high reliability. Therefore, the problem of relatively low reliability of video interaction in the prior art can be improved.

[0072] It should be noted that for step S120, the specific method of determining the target real-time monitoring video is not limited and can be selected according to actual needs.

[0073] For example, in the first alternative embodiment, the video surveillance device may be a fixedly deployed camera or other device to perform video surveillance on a fixed area. For another example, in the second alternative embodiment, the video surveillance device may be a drone-mounted device to move in an area and perform corresponding video surveillance.

[0074] In a first alternative implementation, in order to avoid problems such as waste of resources that are prone to occur in video surveillance to a certain extent, the above-mentioned step S120 may further include step S121 and step S122, and the specific content of each step is as follows.

[0075] Step S121 : determining each video surveillance device deployed in the area of ​​interest reflected by the target location information to form at least one target video surveillance device.

[0076] In the embodiment of the present application, each video surveillance device deployed in the area of ​​interest reflected by the target location information can be determined to form at least one target video surveillance device. That is, since the video surveillance devices are fixedly deployed, each video surveillance device deployed in the area of ​​interest reflected by the target location information (such as all video surveillance devices of a certain scenic spot in a certain scenic spot) can be directly used as a target video surveillance device.

[0077] Step S122, respectively controlling each of the target video surveillance devices to start and perform video surveillance, and obtaining a target real-time surveillance video corresponding to each of the target video surveillance devices.

[0078] In the embodiment of the present application, after the target video surveillance device is determined, each of the target video surveillance devices can be controlled to start and perform video surveillance, and the target real-time surveillance video corresponding to each of the target video surveillance devices can be obtained. In other words, when there is no user demand, the video surveillance device can be turned off, and the video surveillance device can be turned on only when there is a corresponding user demand, so as to avoid consuming the resources of the video surveillance device by turning on the video surveillance device when there is no user demand.

[0079] It is understandable that, in the above step S121, the specific method of determining each video surveillance device deployed in the area of ​​interest reflected by the target location information is not limited and can be selected according to actual needs. For example, in an alternative implementation, in order to further reduce the consumption of device resources, the above step S121 can further include step S121a, step S121b and step S121c, and the specific content of each step is as follows.

[0080] Step S121a, determining each video surveillance device deployed in the area of ​​interest reflected by the target location information to form at least one candidate video surveillance device.

[0081] In the embodiment of the present application, each video surveillance device deployed in the area of ​​interest reflected by the target location information can be determined to form at least one candidate video surveillance device. In other words, each video surveillance device in the corresponding area can be used as a candidate video surveillance device.

[0082] Step S121b: When the number of the at least one candidate video surveillance device is less than or equal to a predetermined reference number, use the at least one candidate video surveillance device as a target event monitoring device.

[0083] In an embodiment of the present application, after determining at least one candidate video surveillance device, when the number of the at least one candidate video surveillance device is less than or equal to a predetermined reference number, a corresponding candidate video surveillance device can be used as a target event monitoring device. That is, when the number of the candidate video surveillance devices is small, there is no problem of excessive consumption of device resources, so each candidate video surveillance device can be directly used as a target event monitoring device. In addition, the specific value of the reference number is not limited, such as 1, 2, 3, 4, etc.

[0084] Step S121c, when the number of the at least one candidate video surveillance device is greater than the reference number, based on a predetermined device screening rule, at least one candidate video surveillance device is determined from the at least one candidate video surveillance device as a target event monitoring device.

[0085] In the embodiment of the present application, when the number of the at least one candidate video surveillance device is greater than the reference number, based on the predetermined device screening rule, at least one candidate video surveillance device is determined from the at least one candidate video surveillance device as the target event monitoring device. That is, when the number of the candidate video surveillance devices is large, the at least one candidate video surveillance device can be further screened, so that the number of enabled video surveillance devices can be reduced to a certain extent, so as to improve the problem of excessive device resource consumption.

[0086] It is understandable that, in the above step S121c, the specific method of determining at least one candidate video surveillance device as the target event monitoring device from the at least one candidate video surveillance device is not limited and can be selected according to actual needs. For example, in an alternative implementation, in order to improve the problem of excessive consumption of device resources while also taking into account the adaptation to the actual needs of users to ensure the stickiness of users to the corresponding platform, the above step S121c can further include the following specific implementation content:

[0087] First, when the number of the at least one candidate video surveillance device is greater than the reference number, the region description information of the local surveillance area corresponding to each candidate video surveillance device may be obtained, wherein the region description information includes video and / or text, such as a video and text of a scenic spot introduction, and the video may be obtained by monitoring the scenic spot;

[0088] Secondly, the target user demand information sent by the target user through the target terminal device can be obtained, wherein the target user demand information is used to reflect the content of interest of the target user, such as the type of scenic spots that the target user likes, such as lakes, buildings, etc.;

[0089] Then, for each of the candidate video surveillance devices, the degree of match between the region description information of the local surveillance region corresponding to the candidate video surveillance device and the target user demand information can be determined; illustratively, the region description information and the target user demand information can be respectively feature mined through a trained neural network to obtain corresponding region description information features and user demand information features, and then the feature similarity (such as cosine similarity, etc.) between the region description information features and the user demand information features can be calculated as the corresponding degree of match. Specifically, for video frames, convolution processing and self-attention processing can be performed, and for text, word embedding processing and self-attention processing can be performed. The specific implementation process can refer to the relevant prior art;

[0090] Finally, each candidate video surveillance device with a matching degree greater than or equal to a predetermined reference matching degree can be used as a target event monitoring device, or a target number of candidate video surveillance devices with the largest matching degrees can be used as target event monitoring devices; the specific values ​​of the reference matching degree and the target number are not restricted and can be configured according to actual needs.

[0091] In a second alternative implementation, in order to ensure that the obtained target real-time monitoring video can better characterize the corresponding region of interest, the above-mentioned step S120 may further include step S123 and step S124, and the specific content of each step is described as follows.

[0092] Step S123, determining the video surveillance equipment deployed in the area of ​​interest reflected by the target location information to form a target video surveillance equipment.

[0093] In an embodiment of the present application, the video surveillance equipment deployed in the area of ​​interest reflected by the target location information can be determined to form a target video surveillance equipment. Among them, the video surveillance equipment belongs to a drone-mounted device. That is to say, generally speaking, since drone-mounted equipment has the characteristic of moving within a certain area, the number requirement for the deployment of drone-mounted equipment is not high. For example, one drone-mounted equipment can be deployed in a scenic spot, or multiple drone-mounted equipment can be deployed in a scenic spot to accommodate concurrent video interaction needs. Based on this, the video surveillance equipment deployed in the area of ​​interest reflected by the target location information can be directly used as the target video surveillance equipment.

[0094] Step S124, controlling the target video monitoring device to move in the area of ​​interest reflected by the target position information, and performing video monitoring during the movement to obtain a corresponding target real-time monitoring video.

[0095] In the embodiment of the present application, after the target video surveillance device is determined, the target video surveillance device can be controlled to move in the area of ​​interest reflected by the target position information, and video surveillance is performed during the movement to obtain the corresponding target real-time surveillance video. In other words, the target video surveillance device can be used to perform video surveillance on multiple local areas in the corresponding area, so that the obtained target real-time surveillance video has rich video content.

[0096] It is understandable that, in the above step S124, the specific manner of controlling the target video monitoring device to move in the region of interest reflected by the target position information is not limited, and can be selected accordingly according to actual needs. For example, in an alternative implementation, in order to make the acquired target real-time monitoring video more adaptable to the actual needs of the user on the basis of having rich video content, the above step S124 can further include step S124a, step S124b and step S124c, and the specific contents of each step are as follows.

[0097] Step S124a, obtaining target user demand information sent by the target user through the target terminal device.

[0098] In the embodiment of the present application, the target user demand information sent by the target user through the target terminal device can be obtained, wherein the target user demand information is used to reflect the content of interest of the target user, such as favorite food, attractions, etc.

[0099] Step S124b: Based on the target user demand information, determine a target motion path in the region of interest reflected by the target position information.

[0100] In an embodiment of the present application, after obtaining the target user demand information, a target motion path can be determined in the region of interest reflected by the target position information based on the target user demand information. The target motion path includes at least a starting point and an end point. That is, the region in the target motion path is where the target user may be more interested.

[0101] Step S124c, controlling the target video monitoring device to move along the target motion path in the area of ​​interest reflected by the target position information, and performing video monitoring through the target video monitoring device during the movement to obtain a corresponding target real-time monitoring video.

[0102] In an embodiment of the present application, after determining the target motion path, the target video surveillance device can be controlled to move along the target motion path in the area of ​​interest reflected by the target position information, and video surveillance can be performed through the target video surveillance device during the movement to obtain the corresponding target real-time surveillance video.

[0103] It can be understood that, in the above-mentioned step S124b, the specific method of determining the target motion path in the area of ​​interest reflected by the target position information is not limited, and can be selected accordingly according to actual needs. For example, in an alternative embodiment, in order to ensure the reliability of determining the target motion path, the above-mentioned step S124b can further include step b1, step b2, step b3, step b4, step b5 and step b6, and the specific content of each step is as follows.

[0104] Step b1, based on the target user demand information, determine each historical user having demand correlation with the target user to form at least one related historical user.

[0105] In the embodiment of the present application, each historical user having demand correlation with the target user can be determined based on the target user demand information to form at least one related historical user (i.e., the corresponding user demand information is correlated with the target user demand information). The historical user refers to a user who has historically interacted with the electronic device through real-time video.

[0106] Step b2, obtaining historical real-time monitoring videos formed in history by each of the relevant historical users, and forming at least one historical real-time monitoring video.

[0107] In an embodiment of the present application, after the relevant historical users are determined, the historical real-time monitoring videos formed historically by each of the relevant historical users (i.e., historically used as corresponding target real-time monitoring videos) can be obtained to form at least one historical real-time monitoring video.

[0108] Step b3, mining and aggregating multimodal features of the at least one historical real-time monitoring video and the target user demand information to form multimodal user demand features corresponding to the target user.

[0109] In an embodiment of the present application, after obtaining the historical real-time surveillance video, multimodal feature mining and aggregation can be performed on the at least one historical real-time surveillance video and the target user demand information to form a multimodal user demand feature corresponding to the target user. The multimodal user demand feature is used to reflect the semantic information possessed by the at least one historical real-time surveillance video and the target user demand information. For example, for video frames, convolution processing can be performed to obtain corresponding convolution features, and then, for text, word embedding processing can be performed to obtain corresponding embedding features. In this way, the convolution features and the embedding features can be spliced, or, based on the convolution features, the embedded features can be cross-attention processed to obtain the corresponding multimodal user demand features.

[0110] Step b4, performing path planning processing on the area of ​​interest reflected by the target position information to form multiple candidate motion paths, and, for each of the candidate motion paths, respectively obtaining the node description information of each path node in the candidate motion path, and combining the node description information of each path node to form the path description information corresponding to the candidate motion path.

[0111] In an embodiment of the present application, a path planning process can be performed on the area of ​​interest reflected by the target location information to form multiple candidate motion paths, and for each candidate motion path, node description information (such as scenic spot introduction, etc.) of each path node in the candidate motion path is obtained respectively, and the node description information of each path node is combined to form the path description information corresponding to the candidate motion path. The node description information includes video and / or text, i.e., video introduction or text introduction, etc.

[0112] Step b5: performing feature mining on the path description information corresponding to each candidate motion path to form a path description feature corresponding to each candidate motion path.

[0113] In an embodiment of the present application, after obtaining the path description information, feature mining can be performed on the path description information corresponding to each candidate motion path to form a path description feature corresponding to each candidate motion path. Specifically, convolution processing can be performed on the video frame to obtain the corresponding convolution feature, and then, word embedding processing can be performed on the text to obtain the corresponding embedding feature, so that the convolution feature and the embedding feature can be spliced, or the embedded feature can be cross-attention processed based on the convolution feature to obtain the path description feature.

[0114] Step b6, respectively determining the similarity between the path description feature corresponding to each of the candidate motion paths and the multimodal user demand feature, and determining the candidate motion path with the maximum similarity as the target motion path.

[0115] In the embodiment of the present application, after obtaining the path description feature and the multimodal user demand feature, the similarity between the path description feature corresponding to each candidate motion path and the multimodal user demand feature (such as the cosine similarity between the features) can be determined respectively, and the candidate motion path with the maximum similarity is determined as the target motion path. In addition, in the embodiment of the present application, the specific form of each feature is not limited. For example, it can be a vector. Specifically, for the text "Tianzi Mountain is famous for its natural landscapes such as strange peaks, sea of ​​clouds, sunrise, and waterfalls", the following word embedding features corresponding to each word can be obtained by performing word segmentation embedding processing:

[0116] Tianzi Mountain: [0.23, -0.12, 0.85, 0.09, -0.67, 0.54, 0.33, -0.44, 0.71, 0.11, 0.88, -0.12, -0.43, ..., 0.56];

[0117] With: [-0.15, 0.21, 0.76, -0.02, 0.34, -0.56, 0.29, 0.33, -0.23, 0.57, -0.45, 0.61, 0.05, ..., -0.11];

[0118] Odd peaks: [0.41, -0.67, 0.53, 0.22, 0.15, 0.78, -0.34, -0.59, 0.21, -0.44, 0.39, -0.25, 0.12, ..., 0.67];

[0119] Sea of ​​clouds: [0.37, 0.54, 0.65, -0.21, 0.56, -0.14, 0.68, 0.03, 0.44, -0.31, 0.72, 0.19, -0.22, ..., -0.58];

[0120] Sunrise: [0.29, 0.11, 0.78, 0.23, -0.11, 0.91, -0.02, 0.57, 0.34, 0.65, 0.41, -0.55, 0.67, ..., -0.23];

[0121] waterfall: [0.62, 0.33, -0.09, 0.56, 0.79, 0.21, 0.54, -0.29, 0.63, 0.12, -0.67, 0.41, -0.13, ..., 0.76];

[0122] etc.: [-0.32, 0.48, 0.21, 0.67, -0.09, -0.17, 0.54, 0.77, -0.14, 0.33, 0.12, 0.43, 0.58, ..., 0.29];

[0123] Nature: [0.68, 0.53, -0.23, 0.34, 0.74, 0.11, 0.44, -0.32, -0.07, 0.25, 0.33, 0.41, 0.58, ..., 0.09];

[0124] Landscape: [0.45, 0.39, 0.57, 0.23, -0.21, 0.12, 0.88, 0.32, 0.47, -0.31, 0.51, 0.67, 0.19, ..., -0.64];

[0125] Notable: [0.22, 0.17, -0.12, 0.56, 0.19, 0.82, -0.33, 0.27, 0.75, 0.39, -0.23, 0.51, 0.41, ..., 0.03].

[0126] It can be understood that, in the above step b1, the specific method of determining each historical user having demand correlation with the target user based on the target user demand information is not limited, and can be selected accordingly according to actual needs. For example, in an alternative implementation, in order to ensure that the determined relevant historical users have high reliability, so that the needs of the target user can be characterized based on the historical real-time monitoring videos of the relevant historical users, the above step b1 can further include the following specific implementation contents:

[0127] First, a historical user relationship network constructed based on each historical user and the historical user demand information corresponding to each historical user can be determined (in the historical user relationship network, there is demand correlation between two connected historical users), and, based on the target user and the target user demand information, the historical user relationship network is first updated to form a first updated user relationship network, wherein, in the first updating process, each historical user corresponding to the historical user demand information whose relevance to the target user demand information (which can be the semantic similarity between texts, etc.) is greater than or equal to a preset relevance (which can be configured accordingly according to actual conditions, such as 0.5, 0.6, 0.7, etc.) is connected to the target user to indicate that the historical user has demand correlation with the target user, and combined with Figure 3 As shown;

[0128] Secondly, feature mining (such as word embedding processing and self-attention processing) can be performed on each of the to-be-processed user demand information, and the to-be-processed user demand features corresponding to each of the to-be-processed user demand information can be output, wherein each of the to-be-processed user demand information belongs to the historical user demand information or the target user demand information;

[0129] Then, for each user to be processed in the first updated user relationship network, a local user relationship network centered on the user to be processed is determined in the first updated user relationship network, wherein each of the users to be processed belongs to the historical user or the target user; illustratively, the degree of connectivity between other users to be processed in the local user relationship network and the central user to be processed needs to be less than or equal to a preset value, such as 0, 1, 2, 3, etc., wherein the connectivity may refer to the number of users to be processed included in the shortest connection path between two users to be processed, Figure 3 For example, when the target user is the center, other users to be processed with a connectivity of 0 include historical user 3, historical user 5, and historical user 8, other users to be processed with a connectivity of 1 include historical user 2 and historical user 7, other users to be processed with a connectivity of 2 include historical user 4, and other users to be processed with a connectivity of 3 include historical user 1 and historical user 6. Thus, if the preset value is 1, the local user relationship network centered on the target user is as follows: Figure 4 As shown;

[0130] Afterwards, a random walk can be performed in the first updated user relationship network to form a plurality of random walk paths (exemplarily, each random walk path is determined, as long as any two random walk paths are not completely the same, or a number of random walk paths can be randomly determined), and the plurality of random walk paths are randomly serialized (i.e., the plurality of random walk paths are arbitrarily sorted) to form a corresponding target path sequence, wherein the path end point of each of the random walk paths is the target user, such as Figure 3 "Historical user 1, historical user 4, historical user 2, historical user 3, target user";

[0131] Further, the currently traversed user to be processed can be determined according to the order of the multiple random walk paths in the target path sequence and the direction from the path start point to the path end point in the random walk path, and, based on the current user demand features of the currently traversed user to be processed corresponding to each user to be processed in the local user relationship network of the currently traversed user to be processed, the current user demand features of the currently traversed user to be processed corresponding to the current user demand features to be processed are associated with mining to obtain the corresponding associated user demand features, and the associated user demand features are used as new current user demand features to be processed, and after the current user demand features to be processed corresponding to the target user are associated with mining in the last random walk path, the first updated user relationship network is secondly updated based on the feature similarity between the current user demand features to be processed corresponding to each user to be processed to form a second updated user relationship network; illustratively, two users to be processed whose feature similarity is greater than or equal to the target similarity can be connected together, and the specific value of the target similarity can be configured according to actual needs, such as 0.5, 0.6, 0.7, etc.; for association mining, with Figure 4Taking an example to illustrate, in the first step, based on the current user demand characteristics to be processed corresponding to historical user 2, the current user demand characteristics to be processed corresponding to historical user 3 are cross-attention processed to obtain the new current user demand characteristics to be processed corresponding to historical user 3, and, based on the current user demand characteristics to be processed corresponding to historical user 7, the current user demand characteristics to be processed corresponding to historical user 5 are cross-attention processed to obtain the new current user demand characteristics to be processed corresponding to historical user 5, and, based on the current user demand characteristics to be processed corresponding to historical user 7, the current user demand characteristics to be processed corresponding to historical user 8 are cross-attention processed to obtain the new current user demand characteristics to be processed corresponding to historical user 8; in the second step, based on the current user demand characteristics to be processed corresponding to historical user 3 Demand characteristics, cross-attention processing is performed on the current user demand characteristics to be processed corresponding to the target user to obtain the new current user demand characteristics to be processed corresponding to the target user, and, based on the current user demand characteristics to be processed corresponding to the historical user 5, the current user demand characteristics to be processed corresponding to the target user are cross-attention processed to obtain the new current user demand characteristics to be processed corresponding to the target user, and, based on the current user demand characteristics to be processed corresponding to the historical user 8, the current user demand characteristics to be processed corresponding to the target user are cross-attention processed to obtain the new current user demand characteristics to be processed corresponding to the target user; In the third step, the new current user demand characteristics to be processed obtained by the three cross-attention processing can be averaged to obtain the actual current user demand characteristics to be processed of the target user;

[0132] Finally, each historical user that has a connection relationship with the target user can be determined in the second updated user relationship network to form at least one related historical user; that is, each historical user that has a connection relationship with the target user in the second updated user relationship network can be used as a related historical user of the target user.

[0133] Combination Figure 5 The embodiment of the present application also provides a video interaction device based on user needs that can be applied to the above electronic device. The video interaction device based on user needs may include a location information determination module, a monitoring video determination module, and a monitoring video transmission module.

[0134] In detail, the location information determination module can be used to obtain the target interaction request information sent by the target user through the corresponding target terminal device, and parse the target interaction request information to obtain the target location information corresponding to the target user, wherein the target location information is used to reflect the area of ​​interest of the target user. In the embodiment of the present application, the location information determination module can be used to perform Figure 2 As shown in step S110, the relevant contents of the location information determination module can refer to the above description of step S110.

[0135] In detail, the monitoring video determination module can be used to determine the target real-time monitoring video based on the target location information, wherein the target real-time monitoring video is obtained by real-time video monitoring of the area of ​​interest by a video monitoring device deployed in the area of ​​interest reflected by the target location information. Figure 2 As shown in step S120, the relevant contents of the monitoring video determination module can refer to the above description of step S120.

[0136] In detail, the monitoring video transmission module can be used to transmit the target real-time monitoring video to the target terminal device corresponding to the target user, so as to complete the real-time video interaction between the electronic device and the target terminal device. Figure 2 As shown in step S130, the relevant contents of the monitoring video transmission module can refer to the above description of step S130.

[0137] In an embodiment of the present application, corresponding to the above-mentioned video interaction method based on user needs applied to the electronic device, a computer-readable storage medium is also provided, in which a computer program is stored, and when the computer program is run, each step of the video interaction method based on user needs is executed.

[0138] Among them, the steps executed when the aforementioned computer program is running will not be described one by one here, and reference may be made to the above explanation of the video interaction method based on user needs.

[0139] In summary, the video interaction method, device and equipment based on user needs provided by the present application first obtain the target interaction request information sent by the target user through the corresponding target terminal device, and parse the target interaction request information to obtain the target location information corresponding to the target user; secondly, based on the target location information, determine the target real-time monitoring video; then, transmit the target real-time monitoring video to the target terminal device corresponding to the target user to complete the real-time video interaction between the electronic device and the target terminal device. Based on the above content, after analyzing the target location information of the area of ​​interest of the target user, the video monitoring device deployed in the area of ​​interest reflected by the target location information can be used to perform real-time video monitoring of the area of ​​interest, so that the obtained target real-time monitoring video has high real-time performance, thereby ensuring that the obtained relevant information has high reliability, and therefore, the problem of relatively low reliability of video interaction existing in the prior art can be improved.

[0140] In several embodiments provided in the embodiments of the present application, it should be understood that the disclosed device and method can also be implemented in other ways. The device and method embodiments described above are merely schematic, for example, the flowcharts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the device, method and computer program product according to multiple embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, a program segment or a part of a code, and the module, a program segment or a part of a code contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, and the combination of boxes in the block diagram and / or flowchart can be implemented with a dedicated hardware-based system that performs a specified function or action, or can be implemented with a combination of dedicated hardware and computer instructions.

[0141] In addition, the functional modules in the various embodiments of the present application may be integrated together to form an independent part, or each module may exist separately, or two or more modules may be integrated to form an independent part.

[0142] If the function is implemented in the form of a software function module and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or the part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, an electronic device, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a disk or an optical disk. It should be noted that, in this article, the term "include", "include" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements includes not only those elements, but also includes other elements that are not explicitly listed, or also includes elements inherent to such a process, method, article or device. Without more constraints, an element defined by the phrase "comprising a..." does not exclude the existence of other identical elements in the process, method, article or apparatus comprising the element.

[0143] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A video interaction method based on user needs, characterized in that: Applied to electronic equipment, wherein the video interaction method based on user demand includes: Acquire target interaction request information sent by a target user through a corresponding target terminal device, and parse the target interaction request information to obtain target location information corresponding to the target user, wherein the target location information is used to reflect the area of ​​interest of the target user; Based on the target location information, determine a target real-time monitoring video, wherein the target real-time monitoring video is obtained by performing real-time video monitoring of the area of ​​interest by a video monitoring device deployed in the area of ​​interest reflected by the target location information; The target real-time monitoring video is transmitted to the target terminal device corresponding to the target user to complete the real-time video interaction between the electronic device and the target terminal device.

2. The video interaction method based on user needs according to claim 1, characterized in that: The step of determining the target real-time monitoring video based on the target location information includes: Determine each video surveillance device deployed in the area of ​​interest reflected by the target location information to form at least one target video surveillance device; Each of the target video monitoring devices is controlled to start and perform video monitoring, and a target real-time monitoring video corresponding to each of the target video monitoring devices is obtained.

3. The video interaction method based on user needs according to claim 2, characterized in that: The step of determining each video surveillance device deployed in the area of ​​interest reflected by the target location information to form at least one target video surveillance device includes: Determine each video surveillance device deployed in the area of ​​interest reflected by the target location information to form at least one candidate video surveillance device; When the number of the at least one candidate video surveillance device is less than or equal to a predetermined reference number, taking the at least one candidate video surveillance device as a target event monitoring device; When the number of the at least one candidate video surveillance device is greater than the reference number, at least one candidate video surveillance device is determined as a target event monitoring device from the at least one candidate video surveillance device based on a predetermined device screening rule.

4. The video interaction method based on user needs according to claim 3 is characterized in that: The step of determining at least one candidate video surveillance device as a target event monitoring device from the at least one candidate video surveillance device based on a predetermined device screening rule when the number of the at least one candidate video surveillance device is greater than the reference number includes: When the number of the at least one candidate video surveillance device is greater than the reference number, obtaining region description information of the local surveillance area corresponding to each of the candidate video surveillance devices, wherein the region description information includes video and / or text; Acquire target user demand information sent by the target user through the target terminal device, wherein the target user demand information is used to reflect the content of interest to the target user; For each of the candidate video surveillance devices, determining a degree of match between the area description information of the local surveillance area corresponding to the candidate video surveillance device and the target user demand information; Each candidate video surveillance device with a matching degree greater than or equal to a predetermined reference matching degree is taken as a target event monitoring device, or a target number of candidate video surveillance devices with the largest matching degrees are taken as target event monitoring devices.

5. The video interaction method based on user demand according to claim 1, characterized in that: The step of determining the target real-time monitoring video based on the target location information includes: Determine the video surveillance equipment deployed in the area of ​​interest reflected by the target location information to form a target video surveillance equipment, wherein the video surveillance equipment is an airborne equipment of a drone; The target video monitoring device is controlled to move in the area of ​​interest reflected by the target position information, and video monitoring is performed during the movement to obtain a corresponding target real-time monitoring video.

6. The video interaction method based on user demand according to claim 5, characterized in that: The step of controlling the target video monitoring device to move in the area of ​​interest reflected by the target position information and performing video monitoring during the movement to obtain the corresponding target real-time monitoring video includes: Acquire target user demand information sent by the target user through the target terminal device, wherein the target user demand information is used to reflect the content of interest to the target user; Based on the target user demand information, determining a target motion path in the region of interest reflected by the target position information, wherein the target motion path includes at least a starting point and an end point; The target video monitoring device is controlled to move along the target motion path in the area of ​​interest reflected by the target position information, and video monitoring is performed through the target video monitoring device during the movement to obtain a corresponding target real-time monitoring video.

7. The video interaction method based on user demand according to claim 6, characterized in that: The step of determining the target motion path in the region of interest reflected by the target position information based on the target user demand information comprises: Determine each historical user having demand correlation with the target user based on the target user demand information to form at least one related historical user, wherein the historical user refers to a user who has historically performed real-time video interaction with the electronic device; Acquire historical real-time monitoring videos formed in the history of each of the relevant historical users to form at least one historical real-time monitoring video; Performing multimodal feature mining and aggregation on the at least one historical real-time monitoring video and the target user demand information to form a multimodal user demand feature corresponding to the target user, wherein the multimodal user demand feature is used to reflect the semantic information possessed by the at least one historical real-time monitoring video and the target user demand information; Performing path planning processing on the area of ​​interest reflected by the target position information to form a plurality of candidate motion paths, and, for each of the candidate motion paths, respectively obtaining node description information of each path node in the candidate motion path, and combining the node description information of each path node to form path description information corresponding to the candidate motion path, wherein the node description information includes video and / or text; Performing feature mining on the path description information corresponding to each of the candidate motion paths to form a path description feature corresponding to each of the candidate motion paths; The similarity between the path description feature corresponding to each of the candidate motion paths and the multimodal user demand feature is determined respectively, and the candidate motion path with the maximum similarity is determined as the target motion path.

8. The video interaction method based on user demand according to claim 7, characterized in that: The step of determining each historical user having demand correlation with the target user based on the target user demand information to form at least one related historical user includes: Determine a historical user relationship network constructed based on each historical user and the historical user demand information corresponding to each historical user, and, based on the target user and the target user demand information, perform a first update on the historical user relationship network to form a first updated user relationship network, wherein, during the first update process, connect each historical user corresponding to each historical user demand information whose correlation with the target user demand information is greater than or equal to a preset correlation with the target user demand information to indicate that the historical user has demand correlation with the target user; Perform feature mining on each of the to-be-processed user demand information respectively, and output the to-be-processed user demand features corresponding to each of the to-be-processed user demand information, wherein each of the to-be-processed user demand information belongs to the historical user demand information or the target user demand information; For each user to be processed in the first updated user relationship network, a local user relationship network centered on the user to be processed is determined in the first updated user relationship network, wherein each of the users to be processed belongs to the historical user or the target user; Performing random walks in the first updated user relationship network to form a plurality of random walk paths, and randomly serializing the plurality of random walk paths to form a corresponding target path sequence, wherein the path end point of each of the random walk paths is the target user; According to the order relationship of the multiple random walk paths in the target path sequence, and according to the direction from the path starting point to the path ending point in the random walk path, the currently traversed user to be processed is determined, and based on the current user demand features of the currently traversed user to be processed corresponding to each user to be processed in the local user relationship network of the currently traversed user to be processed, the current user demand features of the currently traversed user to be processed corresponding to the current user demand features to be processed are associated with mining to obtain the corresponding associated user demand features, and the associated user demand features are used as new current user demand features to be processed, and after the current user demand features to be processed corresponding to the target user are associated with mining in the last random walk path, based on the feature similarity between the current user demand features to be processed corresponding to each user to be processed, the first updated user relationship network is second updated to form a second updated user relationship network; In the second updated user relationship network, each historical user having a connection relationship with the target user is determined to form at least one related historical user.

9. A video interaction device based on user needs, characterized in that: Applied to electronic equipment, wherein the video interaction device based on user needs comprises: a location information determination module, configured to obtain target interaction request information sent by a target user through a corresponding target terminal device, and parse the target interaction request information to obtain target location information corresponding to the target user, wherein the target location information is used to reflect an area of ​​interest of the target user; A monitoring video determination module, used to determine a target real-time monitoring video based on the target location information, wherein the target real-time monitoring video is obtained by performing real-time video monitoring of the area of ​​interest by a video monitoring device deployed in the area of ​​interest reflected by the target location information; The monitoring video transmission module is used to transmit the target real-time monitoring video to the target terminal device corresponding to the target user to complete the real-time video interaction between the electronic device and the target terminal device.

10. An electronic device, characterized in that: include: Memory for storing computer programs; A processor connected to the memory is used to execute the computer program stored in the memory to implement the video interaction method based on user needs as described in any one of claims 1-8.

Images