Directing method and device, terminal equipment and storage medium
By real-time detection and extraction of characteristic information of moving objects inside and outside the one-way perspective glass area in the recording and broadcasting system, the problem of missed students in the recording and broadcasting system is solved, and effective detection and output of students is achieved.
Patent Information
- Application Number
- CN202310539843.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-12
- Publication Date
- 2024-11-12
AI Technical Summary
The recording and broadcasting system easily misses the student when shooting one-way perspective glass, resulting in the student being blocked after entering the area where the one-way perspective glass is located, and the student cannot output close-up images.
By detecting video data in real time in the camera of the recording and broadcasting system, marking the optical reflection area, extracting feature information of moving objects inside and outside the target area, determining the corresponding object, outputting close-up video data to avoid missed detection.
Ensure that the recording and broadcasting system can detect and output students' close-up video data even after the student enters the one-way perspective glass area, solving the missed detection problem.
Smart Images

Figure CN118945278A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to the field of information processing, and in particular to a directing method, apparatus, terminal device and storage medium. Background Art
[0002] A recording and broadcasting classroom refers to a classroom equipped with a recording and broadcasting system, which is used to record and broadcast live scenes of classroom lectures or handouts. In order to facilitate outsiders to observe the teaching situation in the recording and broadcasting classroom, and to avoid affecting the students in the recording and broadcasting classroom, one-way perspective glass can be set at the back or both sides of the recording and broadcasting classroom. At present, in order to avoid misidentifying the image reflected by the one-way perspective glass as a real person, when the shooting area of the camera of the recording and broadcasting system includes one-way perspective glass, the area occupied by the one-way perspective glass is generally shielded in the shooting area. However, this method will cause the recording and broadcasting system to ignore the student after entering the area occupied by the one-way perspective glass, and the director screen will not be able to output a close-up image of the student.
[0003] In summary, the method of shielding the area occupied by the one-way perspective glass in the recording and broadcasting system in the prior art has the technical problem of easy missed detection. Summary of the invention
[0004] The embodiments of the present invention provide a directing method, an apparatus, a terminal device and a storage medium. After the students enter the area where the one-way transparent glass is located, the embodiments of the present invention can detect the students and output the close-up video data of the students, thereby solving the technical problem that the recording and broadcasting system in the prior art shields the area occupied by the one-way transparent glass, which is prone to missed detection.
[0005] In a first aspect, an embodiment of the present invention provides a directing method, which is applicable to a first camera in a recording and broadcasting system, where the first camera is used to shoot a first area, and the method includes:
[0006] Determine in real time, based on the captured first video data, whether there is a moving first object in a preset target area in the first video data, where the target area corresponds to an optical reflection area in the first area;
[0007] When the first object exists, extracting first feature information of the first object and recording a first moment of movement of the first object;
[0008] Determining, based on the first video data, whether there is a moving second object outside the target area at a first moment;
[0009] When the second object exists, extracting second feature information of the second object;
[0010] Determining whether the first object corresponds to the second object according to the first feature information and the second feature information;
[0011] After determining that the first object corresponds to the second object, close-up video data of the second object is output in real time.
[0012] In a second aspect, an embodiment of the present invention provides a directing method, which is applicable to a second camera in a recording and broadcasting system, where the second camera is used to shoot a second area, and the method includes:
[0013] receiving an auxiliary detection instruction sent by the first camera, wherein the auxiliary detection instruction carries first feature information of the first object and a first moment at which movement of the first object is detected;
[0014] Determine, based on the second video data captured historically, a third object moving in the second area at the first moment;
[0015] extracting third feature information of a third object;
[0016] Determining whether the first object corresponds to the third object according to the first feature information and the third feature information;
[0017] After determining that the first object corresponds to the third object, close-up video data of the third object is output in real time.
[0018] In a third aspect, an embodiment of the present invention provides a broadcast director device, which is applicable to a first camera in a recording and broadcasting system, the first camera is used to shoot a first area, and the device includes:
[0019] A first object determination module is used to determine in real time whether there is a moving first object in a preset target area in the first video data according to the captured first video data, and the target area corresponds to the optical reflection area in the first area;
[0020] A first feature extraction module, used for extracting first feature information of the first object and recording a first moment of movement of the first object when the first object exists;
[0021] A second object determination module, used for determining whether there is a moving second object outside the target area at a first moment according to the first video data;
[0022] A second feature extraction module, used for extracting second feature information of the second object when the second object exists;
[0023] A first determination module, used to determine whether the first object corresponds to the second object according to the first feature information and the second feature information;
[0024] The first close-up output module is used to output the close-up video data of the second object in real time after determining that the first object corresponds to the second object.
[0025] In a fourth aspect, an embodiment of the present invention provides a broadcast director device, which is applicable to a second camera in a recording and broadcasting system, and the second camera is used to shoot a second area, and the device includes:
[0026] An instruction receiving module is used to receive an auxiliary detection instruction sent by the first camera, wherein the auxiliary detection instruction carries first feature information of the first object and a first moment when the movement of the first object is detected;
[0027] A third object determination module, used for determining, based on the second video data captured historically, a third object moving in the second area at a first moment;
[0028] A third feature extraction module, used to extract third feature information of a third object;
[0029] A second determination module, configured to determine whether the first object corresponds to the third object according to the first feature information and the third feature information;
[0030] The second close-up output module is used to output the close-up video data of the third object in real time after determining that the first object corresponds to the third object.
[0031] In a fifth aspect, an embodiment of the present invention provides a terminal device, the terminal device includes a processor and a memory;
[0032] The memory is used to store the computer program and transmit the computer program to the processor;
[0033] The processor is used to execute a directing method as described in the first aspect or the second aspect according to instructions in the computer program.
[0034] In a sixth aspect, an embodiment of the present invention provides a storage medium storing computer executable instructions, which, when executed by a computer processor, are used to execute a directing method as described in the first aspect or the second aspect.
[0035] As described above, the embodiment of the present invention pre-marks the target area corresponding to the optical reflection area in the video data captured by the camera. When it is determined in real time based on the captured video data that there is a moving first object in the target area, the feature information of the first object is extracted and the first moment of movement of the first object is determined. Then, it is determined whether there is a moving second object outside the target area at the first moment. If so, the second feature information of the second object is extracted, and after determining that the first object corresponds to the second object based on the first feature information and the second feature information, the close-up video data of the second object is output in real time. For the scene where one-way perspective glass is set in the recording and broadcasting classroom, the embodiment of the present invention can detect the student and output the close-up video data of the student even after the student enters the area where the one-way perspective glass is located, and the phenomenon of automatically shielding the student to cause missed detection will not occur, which solves the technical problem that the recording and broadcasting system in the prior art shields the area occupied by the one-way perspective glass, which is prone to missed detection. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] Figure 1 A schematic diagram of the structure of a recording classroom provided in an embodiment of the present invention.
[0037] Figure 2 A flowchart of a directing method provided by an embodiment of the present invention.
[0038] Figure 3 This is a schematic diagram of the first camera provided in an embodiment of the present invention photographing the student standing process.
[0039] Figure 4 An overall framework diagram of a directing method provided in an embodiment of the present invention.
[0040] Figure 5 A schematic diagram of a shooting angle of view of a first camera provided in an embodiment of the present invention.
[0041] Figure 6 A flowchart of another directing method provided by an embodiment of the present invention.
[0042] Figure 7 A schematic diagram of the structure of another recording and broadcasting classroom provided in an embodiment of the present invention.
[0043] Figure 8 An overall framework diagram of another directing method provided in an embodiment of the present invention.
[0044] Fig. 9 A flowchart of another directing method provided by an embodiment of the present invention.
[0045] Fig.10 A schematic structural diagram of a broadcast director device provided in an embodiment of the present invention.
[0046] Fig.11 A schematic structural diagram of another broadcast director device provided in an embodiment of the present invention.
[0047] Fig.12 A schematic diagram of the structure of a terminal device provided by an embodiment of the present invention.
[0048] Reference numerals:
[0049] One-way perspective glass 10 , camera 20 , student 30 , portrait 40 . DETAILED DESCRIPTION
[0050] The following description and accompanying drawings fully illustrate the specific embodiments of the present application so that those skilled in the art can practice them. The examples represent possible variations only. Unless explicitly required, separate components and functions are optional, and the order of operation can vary. The parts and features of some embodiments may be included in or replace the parts and features of other embodiments. The scope of the embodiments of the present application includes the entire scope of the claims, and all available equivalents of the claims. In this article, each embodiment may be represented individually or generally by the term "invention", which is only for convenience, and if more than one invention is disclosed in fact, it is not intended to automatically limit the scope of the application to any single invention or inventive concept. In this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, without requiring or implying any actual relationship or order between these entities or operations. Moreover, the term "include", "comprise" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method or device including a series of elements includes not only those elements, but also other elements that are not explicitly listed. The various embodiments are described in a progressive manner herein, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other. As for the structures, products, etc. disclosed in the embodiments, since they correspond to the parts disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the description of the method part.
[0051] like Figure 1 As shown, Figure 1 A structural diagram of a recording and broadcasting classroom provided by an embodiment of the present invention, Figure 1In the recording and broadcasting classroom, a one-way perspective glass 10 is arranged at the rear, and the reflective surface (mirror surface) of the one-way perspective glass 10 needs to face the space to be observed, that is, the interior of the recording and broadcasting classroom when in use. When the outdoors are brighter than the indoors, the mirror surface of the one-way perspective glass 10 is equivalent to a mirror. At this time, the outdoor scenery cannot be seen indoors, but the indoor scenery can be seen clearly outdoors. When the outdoors are darker than the indoors, the outdoor scenery can be seen indoors, and the indoor scenery can also be seen outdoors, and the clarity depends on the intensity of the outdoor illumination. In the recording and broadcasting classroom provided with the one-way perspective glass 10, a camera 20 for shooting students is arranged at point A above the podium, that is, the shooting area of the camera 20 includes the area where the one-way perspective glass 10 is located. When the outdoors are brighter than the indoors, the mirror surface of the one-way perspective glass 10 will reflect the image in the recording and broadcasting classroom, such as the teacher, students, desks and chairs in the recording and broadcasting classroom.
[0052] At present, in order to prevent the camera 20 from mistakenly identifying the image reflected on the mirror surface of the one-way perspective glass 10 as a real person, the area corresponding to the one-way perspective glass 10 is generally shielded within the shooting area of the camera 20, that is, the area corresponding to the one-way perspective glass 10 is not analyzed and identified. However, with this processing method, when the real figure of the student in the recording and broadcasting classroom enters the area where the one-way perspective glass 10 is located, the student will be shielded by the recording and broadcasting system, and the situation of missed detection will occur. For example, when a student in the last row of the recording and broadcasting classroom stands up to answer the teacher's question, the student's upper body will enter the area where the one-way perspective glass 10 is located. At this time, the recording and broadcasting system will automatically shield the student, resulting in missed detection, and the recording and broadcasting system cannot output a close-up image of the student.
[0053] In summary, in order to solve the above technical problems, Figure 2 As shown, an embodiment of the present invention provides a broadcast directing method. Figure 2A flowchart of a broadcast directing method provided by an embodiment of the present invention. The broadcast directing method provided by an embodiment of the present invention can be executed by a terminal device, which can be implemented by software and / or hardware. The terminal device can be composed of two or more physical entities, or can be composed of one physical entity. For example, the terminal device can be a host computer, a computer, a camera, a mobile phone and other devices. In this embodiment, the terminal device is taken as an example of the first camera in the recording and broadcasting system. The camera is a device that converts optical image signals into electrical signals. When the camera shoots an object, the light reflected from the object is collected by the lens of the camera, so that it is focused on the light receiving surface of the camera device, and then the light is converted into electrical energy by the camera device to obtain video data. The broadcast directing method provided by an embodiment of the present invention is applicable to the first camera in the recording and broadcasting system. A data processing module is provided in the first camera. The data processing module has functions such as information processing and program running. For example, the data processing module can be a central processing unit or a microprocessor, etc. The data processing module in the first camera is used to execute the broadcast directing method provided by the embodiment of the present invention.
[0054] In this embodiment, the first camera is used to shoot the first area, wherein the first area can be set in advance by the user, and when the first camera is installed, the first camera is set to face the first area. For example, the user can set the first area to be the area where the students are located, and the first camera is installed on the ceiling of the podium, such as Figure 1 The first camera is positioned at point A shown in the figure, and is set to face the first area when installed, so that the first camera can shoot the area where the students are located. In another embodiment, when the lens direction of the first camera can be freely rotated, the user can also set the first area by controlling the lens direction of the first camera.
[0055] The broadcast directing method provided by the embodiment of the present invention includes:
[0056] Step 101: Determine in real time whether there is a moving first object in a target area preset in the first video data according to the captured first video data, where the target area corresponds to an optical reflection area in the first area.
[0057] In this embodiment, when the first camera is working, the first camera is used to capture the first video data in the first area, and analyze the captured first video data to determine in real time whether there is a moving first object in the preset target area in the first video data, and the target area corresponds to the optical reflection area in the first area. The optical reflection area refers to the area where light changes its propagation direction during propagation and returns to the original material. For example, light will be reflected when it encounters the surface of water, glass and other objects. In this embodiment, the position of the optical reflection area can be pre-marked in the first video data, and the position can be set as the target area.
[0058] In addition, in this embodiment, when the first camera identifies whether there is a moving first object in the target area of the first video data, a background difference method or a means shift algorithm can be used to identify the moving first object in the target area. Specifically, the basic principle of the background difference method is to subtract the current frame in the image sequence from the background reference model (background image) that has been determined or acquired in real time, and calculate the area where the pixel difference between the current frame and the background image exceeds a certain threshold as the moving area, thereby determining the position, outline and size of the moving object in the current frame, which is very suitable for scenes where the camera is stationary. The basic principle of the Meanshift algorithm is to read the first frame of the video, select the ROI (region of interest) area, and generate a histogram. Then, the following operations are performed on each frame in the video: 1. The histogram is reversely projected to the frame. 2. Based on the window position of the previous frame, a new maximum distribution density is found using means shift to generate a new position window. 3. The window is updated until the last frame, thereby detecting the moving object. The specific process of the background difference method or the meansshift algorithm can refer to the prior art, and will not be repeated in this embodiment.
[0059] For example, in Figure 1 In the recording classroom shown, the user can pre-mark the area where the one-way perspective glass 10 is located in the first video data captured by the first camera, and set the area as the target area in the first video data. In addition, when the students in the last row of the recording classroom are close to the one-way perspective glass 10, it may appear in the first video data that the students in the last row are always located in the target area even if they are sitting and not moving. In order to avoid false detection, the scope of the target area can be appropriately reduced in the first video data, so that the target area is slightly smaller than the area occupied by the one-way perspective glass 10 in the first video data, thereby excluding the students in the last row from the target area. After the first camera captures the first video data of the first area, it can analyze the first video data to determine whether there is a moving first object in the target area in the first video data, that is, whether the moving first object is reflected in the one-way perspective glass 10 in the recording classroom at this time.
[0060] Step 102: When a first object exists, extract first feature information of the first object and record a first moment of movement of the first object.
[0061] When the first camera determines that there is a moving first object in the target area in the first video data, the first camera further extracts the first feature information of the first object from the first video data. The first feature information refers to information used to characterize the features of the first object. Exemplarily, in the scene of the recording classroom, since the first object is a student, the first feature information can be at least one of facial feature information, skeleton feature information, and hairstyle feature information. The facial feature information includes the feature information of the face area, the skeleton feature information includes the feature information of the human skeleton, and the hairstyle feature information includes the feature information of the hairstyle of the face. In one embodiment, when extracting the first feature information from the first video data, a pre-trained deep learning neural network can be used to extract the first feature information, such as a convolutional neural network. The specific process can refer to the prior art and will not be repeated in this embodiment. In addition, while extracting the first feature information of the first object, the first camera needs to record the first moment of the movement of the first object.
[0062] Step 103: Determine, based on the first video data, whether there is a moving second object outside the target area at the first moment.
[0063] After extracting the first characteristic information of the first object and recording the first moment, the first camera needs to further determine whether there is a second object moving outside the target area at the first moment based on the captured first video data. It can be understood that the process of determining the second object is similar to the process of determining the first object, and will not be repeated in this embodiment. In addition, it should be further explained that in the scenario of the recording classroom, after the students move, the students may actually enter the target area in the first video data, but because the first camera is generally installed on the ceiling of the recording classroom, and there is a certain angle between the shooting direction of the first camera and the ceiling, such as Figure 3As shown, the installation position of the first camera is point A. When the student moves to the S1 position, the image is first reflected in the one-way perspective glass. After moving to the S2 position, the first camera will capture the student entering the target area. In this case, the first camera will determine the moment when the student moves to the S1 position as the first moment, that is, the student is still outside the target area at the first moment, so the complete student image can still be detected outside the target area at the first moment. In another embodiment, when the angle between the shooting angle of the first camera and the ceiling is too small, the first camera detects the reflection image of the student in the mirror and the student entering the target area in the first video data at the same time. At this time, the first camera can determine the moment when the reflection image of the student in the mirror is detected as the first moment, and obtain dozens of frames of historical video data captured before the first moment. According to the historical video data, it is determined whether there are moving students outside the target area before the first moment, thereby avoiding the failure to detect the complete student outside the target area at the first moment after the student enters the target area in the first video data.
[0064] Step 104: When the second object exists, extract second feature information of the second object.
[0065] When it is confirmed that there is a moving second object outside the target area at the first moment, the first camera needs to further extract second feature information of the second object, wherein the second feature information is information used to characterize the features of the second object. For details, please refer to the description of the first feature information above, which will not be repeated in this embodiment. It can be understood that when there are multiple second objects, it is necessary to extract feature information of each second object.
[0066] Step 105: Determine whether the first object corresponds to the second object based on the first feature information and the second feature information.
[0067] After the second feature information is extracted, it is determined whether the first object corresponds to the second object based on the first feature information of the first object and the second feature information of the second object, that is, whether the first object reflected in the one-way perspective glass is the image of the second object. Specifically, the first camera can calculate the similarity between the first feature information and each second feature information, and determine whether there is a second object corresponding to the first object based on the calculated similarity. In addition, it should be further explained that in the case of Figure 1In the recording classroom shown, since the students are facing away from the one-way perspective glass, the image reflected by the one-way perspective glass is the back image of the students, that is, the first moving object detected in the target area at this time may not have facial feature information. Therefore, when extracting the first feature information and the second feature information, the skeleton feature information or hairstyle feature information of the first object can be extracted as the first feature information, and the skeleton feature information or hairstyle feature information of the second object can be correspondingly extracted as the second feature information, thereby avoiding the situation where the similarity cannot be calculated later.
[0068] Step 106: After determining that the first object corresponds to the second object, output close-up video data of the second object in real time.
[0069] If the first camera determines the second object corresponding to the first object, it can output the close-up video data of the second object in real time, so that the recording and broadcasting system generates the corresponding streaming media file according to the close-up video data in real time. The overall process is as follows: Figure 4 As shown. The close-up video data refers to video data including a close-up picture of a second object. It can be understood that when there are multiple first objects at the same time, the second object corresponding to each first object can be detected respectively, and the close-up video data of multiple second objects can be output simultaneously.
[0070] For example, in a recording classroom, the shooting angle of the first camera is as follows: Figure 5 As shown, in Figure 5 The area corresponding to the one-way perspective glass 10 is the target area. Figure 5 In the example, when the student 30 stands up from his seat, the corresponding portrait 40 will be reflected in the one-way perspective glass 10. When the first camera detects the presence of a moving first object, i.e., the portrait 40, in the one-way perspective glass 10 based on the first video data captured, the first camera can extract the first feature information of the portrait 40. Afterwards, the first camera detects the second object, i.e., the student 30, moving outside the target area based on the first video data, and extracts the second feature information of the student 30 at the same time, and then calculates the similarity between the first feature information and the second feature information, thereby determining whether the portrait 40 is the image of the student 30 in the one-way perspective glass 10. If so, the first camera outputs the close-up video data of the student 30 in real time.
[0071] As described above, the embodiment of the present invention pre-marks the target area corresponding to the optical reflection area in the video data captured by the camera. When it is determined in real time based on the captured video data that there is a moving first object in the target area, the feature information of the first object is extracted and the first moment of movement of the first object is determined. Then, it is determined whether there is a moving second object outside the target area at the first moment. If so, the second feature information of the second object is extracted, and after determining that the first object corresponds to the second object based on the first feature information and the second feature information, the close-up video data of the second object is output in real time. For the scene where a one-way perspective glass is set in the recording and broadcasting classroom, the embodiment of the present invention can detect the student and output the close-up video data of the student even after the student enters the area where the one-way perspective glass is located, and will not automatically shield the student to cause missed detection, thereby solving the technical problem that the recording and broadcasting system in the prior art shields the area occupied by the one-way perspective glass, which is prone to missed detection.
[0072] like Figure 6 As shown, Figure 6 A flowchart of another directing method provided by an embodiment of the present invention is provided. The directing method provided by an embodiment of the present invention is a specific implementation of the above directing method. The method includes:
[0073] Step 201: Determine in real time whether there is a moving first object in a target area preset in the first video data according to the captured first video data, where the target area corresponds to an optical reflection area in the first area.
[0074] Step 202: When a first object exists, extract first feature information of the first object and record a first moment of movement of the first object.
[0075] Step 203: Determine, based on the first video data, whether there is a moving second object outside the target area at the first moment.
[0076] Step 204: When the second object exists, extract second feature information of the second object.
[0077] Step 205: Calculate a first similarity between the first feature information and the second feature information.
[0078] In this embodiment, after obtaining the first feature information and the second feature information, the first camera can calculate the first similarity between the first feature information and each of the second feature information, wherein the first similarity is used to characterize the degree of similarity between the first feature information and the second feature information. In one embodiment, when calculating the first similarity, the first feature information and the second feature information can be represented as two corresponding high-dimensional vectors, and the first similarity between the two high-dimensional vectors is calculated. A common calculation method is to characterize the distance between the two high-dimensional vectors by the cosine distance, and the distance is the first similarity. In another embodiment, the Euclidean distance, Pearson correlation coefficient or cosine similarity between the two high-dimensional vectors can also be calculated as the first similarity of the first feature information and the second feature information. The specific process can refer to the prior art and will not be repeated in this embodiment.
[0079] Step 206: Determine whether the first similarity is greater than a preset similarity threshold.
[0080] After calculating the first similarity between the first feature information and each second feature information, it is necessary to further determine whether the first similarity is greater than a preset similarity threshold, wherein the similarity threshold is a preset critical value of the similarity, for example, the similarity threshold can be preset to 80%.
[0081] Step 207: If yes, determine that the first object corresponds to the second object.
[0082] If it is determined that there is a first similarity greater than a preset similarity threshold, the second object to which the second feature information used to calculate the first similarity belongs can be determined to correspond to the first object. That is, the first object appearing in the optical reflection area is the image of the second object in the optical reflection area. In addition, when there are multiple first similarities greater than the preset similarity threshold, the target first similarity with the largest first similarity can be determined, and the second object to which the second feature information used to calculate the target first similarity belongs can be determined to correspond to the first object.
[0083] Step 208: After determining that the first object corresponds to the second object, output the close-up video data of the second object in real time.
[0084] As mentioned above, although the present invention can output close-up video data of the second object in real time after determining the second object corresponding to the first object in the optical reflection area. However, in actual use, the object imaged in the one-way transparent glass may be located outside the first area that can be photographed by the first camera. For example, when the first object is the image of a teacher on the podium in the one-way transparent glass, the teacher cannot be detected in the first video data, and the existence of the second object cannot be detected at this time. Based on this, an embodiment of the present invention also provides another directing method. In another directing method, the recording and broadcasting system also includes a second camera, and the second camera is used to shoot a second area, and the second area is different from the first area. For example, as Figure 7 As shown, a second camera is also arranged at point B on the rear ceiling of the recording classroom, and the second area photographed by the second camera is the podium area where the teacher is located. After the first camera and the second camera are started, the first camera will actively establish a communication connection with the second camera, and the first camera and the second camera form a camera group to work together. The directing method provided by the embodiment of the present invention also includes:
[0085] Step 209, when the second object does not exist, an auxiliary detection instruction is sent to the second camera, so that the second camera determines, based on the second video data captured historically and the first feature information, that there is a third object corresponding to the first object in the second area at the first moment, and outputs close-up video data of the third object in real time, and the auxiliary detection instruction carries the first feature information and the first moment.
[0086] In this embodiment, when the first camera cannot identify the second object in the first video data, the first camera will send an auxiliary detection instruction to the second camera, the auxiliary detection instruction is used to instruct the second camera to assist the first camera in detecting the object, and the auxiliary detection instruction carries the first feature information of the first object and the first moment of the movement of the first object. After receiving the auxiliary detection instruction, the second camera will obtain the second video data captured historically, determine the third object moving in the second area at the first moment in the second video data, and extract the third feature information of the third object. Then, the second similarity between the third feature information and the first feature information is calculated to determine whether there is a third object corresponding to the first object in the second area at the first moment. The process of determining the third object corresponding to the first object is similar to the process of the first camera determining the second object corresponding to the first object. The specific process can refer to the process of the first camera determining the second object corresponding to the first object, which will not be repeated in this embodiment. In addition, it should be noted that in the recording classroom, after the teacher is imaged on the mirror surface of the one-way reflective glass, the first camera can capture the teacher's facial image in the one-way reflective glass, and the second camera can also capture the teacher's facial image, that is, the facial feature information of the first object can be used as the first feature information, and the facial feature information of the third object can be used as the third feature information. If the second camera determines the third object corresponding to the first object based on the calculated second similarity, the second camera can output the close-up video data of the third object in real time.
[0087] Based on the above embodiment, it also includes:
[0088] Step 210: When receiving the false detection message sent by the second camera, it is determined that the first object is falsely detected; the false detection message is sent by the second camera to the first camera when it is determined that there is no moving third object in the second area at the first moment.
[0089] In one embodiment, when the second camera determines that there is no moving third object in the second area at the first moment based on the second video data, the second camera determines that the first camera has a false detection, and the second camera sends a false detection message to the first camera, and the false detection message is used to notify the first camera of the false detection. After receiving the false detection message sent by the second camera, the first camera determines that the first object is a false detection, and the first camera can re-detect the moving object in the target area in the first video data. The overall process is as follows: Figure 8As shown. In another embodiment, the second camera can also count the number of false detections of the first camera. When the number of false detections reaches a preset number, the recording and broadcasting system needs to update the method for detecting the first object to improve the accuracy of subsequent detections of the first object. For example, when the first camera detects the first object through a pre-trained neural network, the data of this false detection can be added to the training set, and the neural network can be retrained. In addition, when the first camera determines that there is no second object corresponding to the first object based on the first similarity, or the second camera determines that there is no third object corresponding to the first object based on the second similarity, the detection data of this time can also be added to the training set, and the neural network can be retrained to improve the detection accuracy of the subsequent neural network.
[0090] As mentioned above, in the embodiment of the present invention, when the first camera cannot determine the second object corresponding to the first object in the target area, the first camera will send an auxiliary detection instruction to the second camera, and the second camera will determine whether there is a third object corresponding to the first object in the second area at the first moment based on the second video data captured historically. In the embodiment of the present invention, the first camera and the second camera jointly detect the moving object, which can avoid the situation where the imaged object appearing in the one-way perspective glass is located outside the first area that can be photographed by the first camera, resulting in missed detection, thereby improving the accuracy of subsequent detection. In the embodiment of the present invention, for the scene where the one-way perspective glass is set in the recording and broadcasting classroom, after an object enters the area where the one-way perspective glass is located, the recording and broadcasting system can also detect the object and output the close-up video data of the object, which solves the technical problem that the recording and broadcasting system in the prior art shields the area occupied by the one-way perspective glass, which is prone to missed detection.
[0091] like Fig. 9 As shown, Fig. 9 A flow chart of another directing method provided by an embodiment of the present invention is provided. The method is applicable to a second camera in a recording and broadcasting system. The second camera is used to shoot a second area. The method includes:
[0092] Step 301: Receive an auxiliary detection instruction sent by a first camera, where the auxiliary detection instruction carries first feature information of a first object and a first moment when movement of the first object is detected.
[0093] In this embodiment, when the second camera receives the auxiliary detection instruction sent by the first camera, the second camera first parses the first feature information of the first object in the auxiliary detection instruction and the first moment when the movement of the first object is detected.
[0094] Step 302: Determine, based on the second video data captured historically, a third object moving in the second area at the first moment.
[0095] After parsing the first feature information and the first moment, the second camera will further determine the third object moving in the second area at the first moment based on the second video data captured historically. In one embodiment, when the second camera is capturing the second video data, the second camera will identify the moving object in the second video data in real time and record it. After receiving the auxiliary detection instruction, the second camera can directly determine the third object moving at the first moment based on the previously recorded data, without having to re-identify and analyze the moving object in the historical second video data after obtaining the auxiliary detection instruction.
[0096] Step 303: extract third feature information of the third object.
[0097] After the third object is determined, the second camera can extract third feature information of the third object from the second video data.
[0098] Step 304: Determine whether the first object corresponds to the third object based on the first feature information and the third feature information.
[0099] After extracting the third feature information, the second camera needs to calculate the second similarity between the first feature information and the third feature information according to the first feature information and the third feature information, and determine whether the first object corresponds to the third object according to the second similarity. It can be understood that the process of calculating the second similarity in this embodiment is similar to the process of calculating the first similarity, and will not be repeated in this embodiment.
[0100] Step 305: After determining that the first object corresponds to the third object, output close-up video data of the third object in real time.
[0101] If the second camera determines a third object corresponding to the first object based on the calculated second similarity, the second camera also outputs close-up video data of the third object in real time, so that the recording and broadcasting system generates a corresponding streaming media file based on the close-up video data in real time.
[0102] Based on the above embodiment, it also includes:
[0103] Step 306: When it is determined that there is no moving third object in the second area at the first moment, a false detection message is sent to the first camera.
[0104] In another embodiment, if the second computer determines that there is no moving third object in the second area at the first moment, the second camera determines that the first camera has a false detection, and the second camera sends a false detection message to the first camera to inform the first camera of the false detection.
[0105] As described above, after receiving the auxiliary detection instruction sent by the first camera, the second camera in the embodiment of the present invention will determine whether there is a third object corresponding to the first object in the second area at the first moment according to the second video data captured historically. If there is a third object, the close-up video data of the third object is output in real time. If there is no moving third object, a false detection message is sent to the first camera. The embodiment of the present invention detects moving objects by combining the first camera and the second camera, and can detect moving objects in the recording and broadcasting classroom from multiple angles, avoiding the situation where the objects imaged in the one-way perspective glass are located outside the area that can be photographed by a single camera, resulting in missed detection, thereby improving the accuracy of subsequent detection. For the scene in which the one-way perspective glass is set in the recording and broadcasting classroom, the embodiment of the present invention can detect the object and output the close-up video data of the object after an object enters the area where the one-way perspective glass is located, thereby solving the technical problem that the recording and broadcasting system in the prior art shields the area occupied by the one-way perspective glass, which is prone to missed detection.
[0106] like Fig.10 As shown, Fig.10 The present invention is a structural schematic diagram of a broadcast director provided in an embodiment of the present invention. The broadcast director provided in an embodiment of the present invention is applicable to a first camera in a recording and broadcasting system. The first camera is used to shoot a first area. The broadcast director includes:
[0107] The first object determination module 401 is used to determine in real time whether there is a moving first object in a preset target area in the first video data according to the captured first video data, and the target area corresponds to the optical reflection area in the first area.
[0108] The first feature extraction module 402 is used to extract first feature information of the first object and record a first moment of movement of the first object when the first object exists.
[0109] The second object determination module 403 is used to determine whether there is a moving second object outside the target area at the first moment according to the first video data.
[0110] The second feature extraction module 404 is used to extract second feature information of the second object when the second object exists.
[0111] The first determination module 405 is used to determine whether the first object corresponds to the second object according to the first feature information and the second feature information.
[0112] The first close-up output module 406 is used to output the close-up video data of the second object in real time after determining that the first object corresponds to the second object.
[0113] Based on the above embodiment, the first determining module 405 includes:
[0114] The similarity calculation submodule is used to calculate the first similarity between the first feature information and the second feature information.
[0115] The similarity determination submodule is used to determine whether the first similarity is greater than a preset similarity threshold.
[0116] The corresponding object determination submodule is used to determine that the first object corresponds to the second object if the first similarity is greater than a preset similarity threshold.
[0117] On the basis of the above embodiment, the recording and broadcasting system further includes a second camera, the second camera is used to shoot a second area, and the second area is different from the first area, and the broadcast director device further includes:
[0118] The auxiliary detection module is used to send an auxiliary detection instruction to the second camera when the second object does not exist, so that the second camera determines that when a third object corresponding to the first object exists in the second area at a first moment based on the second video data captured historically and the first feature information, the second camera outputs close-up video data of the third object in real time, and the auxiliary detection instruction carries the first feature information and the first moment.
[0119] Based on the above embodiment, it also includes:
[0120] The message receiving module is used to determine that the first object is misdetected when receiving the misdetection message sent by the second camera; the misdetection message is sent by the second camera to the first camera when it is determined that there is no moving third object in the second area at the first moment.
[0121] The directing device provided in the embodiment of the present invention is included in the terminal equipment and can be used to execute the directing method provided in the above embodiment, and has corresponding functions and beneficial effects.
[0122] It is worth noting that in the embodiment of the above-mentioned director device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.
[0123] like Fig.11 As shown, Fig.11 The present invention is a structural schematic diagram of a broadcast director provided in an embodiment of the present invention. The broadcast director provided in an embodiment of the present invention is applicable to a second camera in a recording and broadcasting system. The second camera is used to shoot a second area. The device includes:
[0124] The instruction receiving module 501 is used to receive an auxiliary detection instruction sent by the first camera, wherein the auxiliary detection instruction carries first feature information of the first object and a first moment when the movement of the first object is detected.
[0125] The third object determining module 502 is configured to determine a third object moving in the second area at a first moment according to the second video data captured historically.
[0126] The third feature extraction module 503 is used to extract third feature information of the third object.
[0127] The second determination module 504 is configured to determine whether the first object corresponds to the third object according to the first feature information and the third feature information.
[0128] The second close-up output module 505 is used to output the close-up video data of the third object in real time after determining that the first object corresponds to the third object.
[0129] Based on the above embodiment, it also includes:
[0130] The message sending module is used to send a false detection message to the first camera when it is determined that there is no moving third object in the second area at the first moment.
[0131] The directing device provided in the embodiment of the present invention is included in the terminal equipment and can be used to execute the directing method provided in the above embodiment, and has corresponding functions and beneficial effects.
[0132] It is worth noting that in the embodiment of the above-mentioned director device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.
[0133] This embodiment also provides a terminal device, such as Fig.12 As shown, the terminal device 60 includes a processor 600 and a memory 601;
[0134] The memory 601 is used to store the computer program 602 and transmit the computer program 602 to the processor 600;
[0135] The processor 600 is used to execute the steps in the above-mentioned embodiment of the directing method according to the instructions in the computer program 602.
[0136] Exemplarily, the computer program 602 may be divided into one or more modules / units, one or more modules / units are stored in the memory 601, and are executed by the processor 600 to complete the present application. One or more modules / units may be a series of computer program instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the computer program 602 in the terminal device 60.
[0137] The terminal device 60 may be a computing device such as a desktop computer, a notebook, a PDA, or a cloud server. The terminal device 60 may include, but is not limited to, a processor 600 and a memory 601. Those skilled in the art will appreciate that Fig.12 It is only an example of the terminal device 60 and does not constitute a limitation of the terminal device 60. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the terminal device 60 may also include input and output devices, network access devices, buses, etc.
[0138] The processor 600 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.
[0139] The memory 601 may be an internal storage unit of the terminal device 60, such as a hard disk or memory of the terminal device 60. The memory 601 may also be an external storage device of the terminal device 60, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the terminal device 60. Further, the memory 601 may also include both an internal storage unit of the terminal device 60 and an external storage device. The memory 601 is used to store computer programs and other programs and data required by the terminal device 60. The memory 601 may also be used to temporarily store data that has been output or is to be output.
[0140] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.
[0141] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.
[0142] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0143] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0144] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, sports hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store computer programs.
[0145] An embodiment of the present invention further provides a storage medium containing computer executable instructions, and when the computer executable instructions are executed by a computer processor, they are used to execute a directing method, the method comprising the following steps:
[0146] Determine in real time, based on the captured first video data, whether there is a moving first object in a preset target area in the first video data, where the target area corresponds to an optical reflection area in the first area;
[0147] When the first object exists, extracting first feature information of the first object and recording a first moment of movement of the first object;
[0148] Determining, based on the first video data, whether there is a moving second object outside the target area at a first moment;
[0149] When the second object exists, extracting second feature information of the second object;
[0150] Determining whether the first object corresponds to the second object according to the first feature information and the second feature information;
[0151] After determining that the first object corresponds to the second object, close-up video data of the second object is output in real time.
[0152] Or including the following steps:
[0153] receiving an auxiliary detection instruction sent by the first camera, wherein the auxiliary detection instruction carries first feature information of the first object and a first moment at which movement of the first object is detected;
[0154] Determine, based on the second video data captured historically, a third object moving in the second area at the first moment;
[0155] extracting third feature information of a third object;
[0156] Determining whether the first object corresponds to the third object according to the first feature information and the third feature information;
[0157] After determining that the first object corresponds to the third object, close-up video data of the third object is output in real time.
[0158] Note that the above are only preferred embodiments of the present invention and the technical principles used. Those skilled in the art will understand that the embodiments of the present invention are not limited to the specific embodiments described herein, and that various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the protection scope of the embodiments of the present invention. Therefore, although the embodiments of the present invention are described in more detail through the above embodiments, the embodiments of the present invention are not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of the embodiments of the present invention, and the scope of the embodiments of the present invention is determined by the scope of the appended claims.
Claims
1. A broadcast directing method, characterized in that: The method is applicable to a first camera in a recording and broadcasting system, where the first camera is used to shoot a first area. The method includes: Determine in real time, based on the captured first video data, whether there is a moving first object in a preset target area in the first video data, where the target area corresponds to an optical reflection area in the first area; When the first object exists, extracting first feature information of the first object and recording a first moment of movement of the first object; determining, according to the first video data, whether there is a moving second object outside the target area at the first moment; When the second object exists, extracting second feature information of the second object; determining, according to the first feature information and the second feature information, whether the first object corresponds to the second object; After determining that the first object corresponds to the second object, close-up video data of the second object is output in real time.
2. A broadcast directing method according to claim 1, characterized in that: The determining, according to the first feature information and the second feature information, whether the first object corresponds to the second object includes: Calculating a first similarity between the first feature information and the second feature information; Determining whether the first similarity is greater than a preset similarity threshold; If so, it is determined that the first object corresponds to the second object.
3. A broadcast directing method according to claim 1, characterized in that: The recording and broadcasting system further includes a second camera, and the second camera is used to shoot a second area, and the second area is different from the first area. The method further includes: When the second object does not exist, an auxiliary detection instruction is sent to the second camera, so that the second camera determines, based on the second video data captured historically and the first feature information, that when a third object corresponding to the first object exists in the second area at the first moment, the second camera outputs close-up video data of the third object in real time, and the auxiliary detection instruction carries the first feature information and the first moment.
4. A broadcast directing method according to claim 3, characterized in that: Also includes: When receiving the false detection message sent by the second camera, determining that the first object is falsely detected; The false detection message is sent by the second camera to the first camera when the second camera determines that there is no moving third object in the second area at the first moment.
5. A broadcast directing method, characterized in that: The method is applicable to a second camera in a recording and broadcasting system, where the second camera is used to shoot a second area, and the method includes: receiving an auxiliary detection instruction sent by a first camera, wherein the auxiliary detection instruction carries first feature information of a first object and a first moment at which movement of the first object is detected; Determine, based on the second video data captured historically, a third object moving in the second area at the first moment; extracting third feature information of the third object; determining, according to the first feature information and the third feature information, whether the first object corresponds to the third object; After determining that the first object corresponds to the third object, close-up video data of the third object is output in real time.
6. A broadcast directing method according to claim 5, characterized in that: Also includes: When it is determined that there is no moving third object in the second area at the first moment, a false detection message is sent to the first camera.
7. A broadcasting device, characterized in that: The device is applicable to a first camera in a recording and broadcasting system, the first camera is used to shoot a first area, and the device includes: A first object determination module, configured to determine in real time, based on the captured first video data, whether there is a moving first object in a target area preset in the first video data, wherein the target area corresponds to an optical reflection area in the first area; A first feature extraction module, configured to extract first feature information of the first object and record a first moment of movement of the first object when the first object exists; A second object determination module, configured to determine, based on the first video data, whether there is a moving second object outside the target area at the first moment; A second feature extraction module, used for extracting second feature information of the second object when the second object exists; A first determining module, configured to determine whether the first object corresponds to the second object according to the first characteristic information and the second characteristic information; The first close-up output module is used to output the close-up video data of the second object in real time after determining that the first object corresponds to the second object.
8. A broadcasting device, characterized in that: The device is applicable to a second camera in a recording and broadcasting system, the second camera is used to shoot a second area, and the device includes: An instruction receiving module, configured to receive an auxiliary detection instruction sent by a first camera, wherein the auxiliary detection instruction carries first feature information of a first object and a first moment at which movement of the first object is detected; A third object determination module, configured to determine a third object moving in the second area at the first moment according to the second video data captured historically; A third feature extraction module, used to extract third feature information of the third object; a second determining module, configured to determine whether the first object corresponds to the third object according to the first feature information and the third feature information; The second close-up output module is used to output the close-up video data of the third object in real time after determining that the first object corresponds to the third object.
9. A terminal device, characterized in that: The terminal device includes a processor and a memory; The memory is used to store a computer program and transmit the computer program to the processor; The processor is used to execute a directing method as described in any one of claims 1-7 according to instructions in the computer program.
10. A storage medium storing computer executable instructions, characterized in that: The computer executable instructions are used to execute a directing method as described in any one of claims 1 to 7 when executed by a computer processor.
