A robot video processing method and storage medium

Abstract

The present invention relates to a robot video processing method and storage medium, the method comprising the following steps: starting a video main thread to collect original video streams in real time through a camera; creating a preset number of data channels according to the number of data sub-threads; when the data sub-thread function is started, opening the corresponding data channel, copying the original video stream, and sending it to the data sub-thread through the corresponding data channel; the data sub-thread converts the received copied original video stream into required video data and then performs video processing. When one or several video scenes need to be used, the data sub-thread copies the original video stream collected by the camera through the corresponding data channel, and converts it into the required video data and then performs video processing, only one camera is needed to simultaneously meet the use of multiple video scenes, no additional cameras need to be added, and no additional costs need to be added.

Description

Technical Field

[0001] The present application relates to the technical field of video processing, and in particular to a robot video processing method. Background Art

[0002] A robot is a machine that performs work automatically. It can accept human commands, execute pre-programmed programs, or act according to principles established by artificial intelligence technology. Robots perform work that replaces or assists human work, such as in manufacturing, construction, or dangerous work.

[0003] However, in the video business scenarios of robots, one camera of existing robots can only meet the needs of one video scene application. If multiple video scenes need to be met, additional cameras need to be added, which will increase additional costs. Summary of the invention

[0004] In view of the above problems, the present application provides a robot video processing method to solve the problem that existing robots need to have multiple cameras to meet the needs of multiple video scenes, which requires additional costs.

[0005] To achieve the above object, the inventor provides a robot video processing method, comprising the following steps:

[0006] Start the video main thread to collect the original video stream in real time through the camera;

[0007] Create a preset number of data channels according to the number of data sub-threads;

[0008] When the data sub-thread function is started, the corresponding data channel is opened, the original video stream is copied, and sent to the data sub-thread through the corresponding data channel;

[0009] The data sub-thread converts the received copied original video stream into the required video data and then performs video processing.

[0010] In some embodiments, the data channel comprises a reserved data channel.

[0011] In some embodiments, the method specifically includes the following steps:

[0012] During initialization, a scheduling container is created to store the video data required by different data sub-threads;

[0013] When receiving a video data request from a user, determine whether the scheduling container is in an idle state;

[0014] If the scheduling container is in idle state, the video main thread is started to collect the original video stream in real time;

[0015] Create a preset number of data channels according to the number of data sub-threads;

[0016] Start the data sub-thread corresponding to the video data requested by the user, copy the original video stream to the scheduling container through the corresponding data channel and convert it into the corresponding type of video data, and set the video data to the working mode;

[0017] If the scheduling container is not in an idle state, determining whether the video data requested by the user is in a working state;

[0018] If the video data requested by the user is not in working state, the data sub-thread corresponding to the video data requested by the user is started, the original video stream is copied to the scheduling container through the corresponding data channel and converted into the corresponding type of video data, and the video data is set to working mode;

[0019] If the video data requested by the user is in a working state, the running state of the video data is fed back to the user.

[0020] In some embodiments, the data sub-threads include a face recognition thread, a remote video thread, a driving record thread, and a photo capture thread.

[0021] In some embodiments, the original video stream is in YUV format;

[0022] The “data sub-thread converts the received copied original video stream into the required video data and then performs video processing” specifically includes the following steps:

[0023] After receiving the copied original video stream, the face recognition thread directly uses the original video stream as the required video data and sends it to the face recognition module;

[0024] After receiving the copied original video stream, the remote video thread converts the original video stream in YUV format into video data in H264 format after data format processing, and sends it to the service platform;

[0025] After receiving the copied original video stream, the driving record thread converts the original video stream in YUV format into video data in H264 format after data format processing, and performs MP4 encapsulation processing on the video data in H264 format to obtain video data in MP4 format, and stores the obtained video data in MP4 format;

[0026] After the photo capture thread receives the copied original video stream, it performs JPG encoding processing to obtain a JPG format image.

[0027] Another technical solution is also provided: a storage medium, wherein the storage medium stores a computer program, and the computer program is executed by a processor to perform the following steps:

[0028] Start the video main thread to collect the original video stream in real time through the camera;

[0029] Create a preset number of data channels according to the number of data sub-threads;

[0030] When the data sub-thread function is started, the corresponding data channel is opened, the original video stream is copied, and sent to the data sub-thread through the corresponding data channel;

[0031] The data sub-thread converts the received copied original video stream into the required video data and then performs video processing.

[0032] In some embodiments, the data channel comprises a reserved data channel.

[0033] In some embodiments, when the computer program is executed by a processor, the following steps are specifically performed:

[0034] During initialization, a scheduling container is created to store the video data required by different data sub-threads;

[0035] When receiving a video data request from a user, determine whether the scheduling container is in an idle state;

[0036] If the scheduling container is in idle state, the video main thread is started to collect the original video stream in real time;

[0037] Create a preset number of data channels according to the number of data sub-threads;

[0038] Start the data sub-thread corresponding to the video data requested by the user, copy the original video stream to the scheduling container through the corresponding data channel and convert it into the corresponding type of video data, and set the video data to the working mode;

[0039] If the scheduling container is not in an idle state, determining whether the video data requested by the user is in a working state;

[0040] If the video data requested by the user is not in working state, the data sub-thread corresponding to the video data requested by the user is started, the original video stream is copied to the scheduling container through the corresponding data channel and converted into the corresponding type of video data, and the video data is set to working mode;

[0041] If the video data requested by the user is in a working state, the running state of the video data is fed back to the user.

[0042] In some embodiments, the data sub-threads include a face recognition thread, a remote video thread, a driving record thread, and a photo capture thread.

[0043] In some embodiments, the original video stream is in YUV format;

[0044] The “data sub-thread converts the received copied original video stream into the required video data and then performs video processing” specifically includes the following steps:

[0045] After receiving the copied original video stream, the face recognition thread directly uses the original video stream as the required video data and sends it to the face recognition module;

[0046] After receiving the copied original video stream, the remote video thread converts the original video stream in YUV format into video data in H264 format after data format processing, and sends it to the service platform;

[0047] After receiving the copied original video stream, the driving record thread converts the original video stream in YUV format into video data in H264 format after data format processing, and performs MP4 encapsulation processing on the video data in H264 format to obtain video data in MP4 format, and stores the obtained video data in MP4 format;

[0048] After the photo capture thread receives the copied original video stream, it performs JPG encoding processing to obtain a JPG format image.

[0049] Different from the prior art, the above technical solution starts the video main thread when the robot is using multiple video scenes, and collects the original video stream in real time through the camera; creates a preset number of data channels according to the number of data sub-threads, and when one or several video scenes need to be used, starts the data sub-thread corresponding to the required video scene, and the data sub-thread copies the original video stream collected by the camera through the corresponding data channel, and converts it into the required video data for video processing. Only one camera is needed to meet the application of multiple video scenes at the same time, without adding additional cameras and without increasing additional costs.

[0050] The above-mentioned records related to the invention content are only an overview of the technical solution of the present application. In order to enable ordinary technicians in the field to more clearly understand the technical solution of the present application, and then implement it according to the text of the specification and the contents recorded in the drawings, and to make the above-mentioned purpose and other purposes, features and advantages of the present application easier to understand, the following is an explanation in combination with the specific implementation mode and drawings of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The drawings are only used to illustrate the principles, implementation methods, applications, characteristics and effects of the specific embodiments of the present application and other related contents, and shall not be considered as limitations of the present application.

[0052] In the drawings of the specification:

[0053] Figure 1 A schematic diagram of a flow chart of the robot video processing method described in the specific implementation method;

[0054] Figure 2 Another schematic flow chart of the robot video processing method described in the specific implementation method;

[0055] Figure 3 A schematic diagram of a workflow of the video data processing subsystem described in the specific implementation method;

[0056] Figure 4 A schematic diagram of a workflow of the video service scheduling subsystem described in the specific implementation method;

[0057] Figure 5 A schematic diagram of the structure of the storage medium described in the specific implementation method.

[0058] The reference numerals in the above drawings are described as follows:

[0059] 510. Storage medium,

[0060] 520. Processor. DETAILED DESCRIPTION

[0061] In order to explain in detail the possible application scenarios, technical principles, specific schemes that can be implemented, and the purposes and effects that can be achieved, the following is a detailed description of the specific embodiments listed in conjunction with the accompanying drawings. The embodiments described herein are only used to more clearly illustrate the technical solutions of the present application, and are therefore only used as examples, and cannot be used to limit the scope of protection of the present application.

[0062] Reference to "embodiment" herein means that the specific features, structures or characteristics described in conjunction with the embodiment may be included in at least one embodiment of the present application. The term "embodiment" appearing in various places in the specification does not necessarily refer to the same embodiment, nor does it particularly limit its independence or association with other embodiments. In principle, in the present application, as long as there is no technical contradiction or conflict, the various technical features mentioned in the embodiments can be combined in any way to form a corresponding implementable technical solution.

[0063] Unless otherwise defined, the technical terms used in this document have the same meanings as those generally understood by those skilled in the art to which this application belongs; the use of relevant terms in this document is only for describing specific embodiments and is not intended to limit this application.

[0064] In the description of this application, the term "and / or" is an expression used to describe the logical relationship between objects, indicating that three relationships may exist, for example, A and / or B, which means: A exists, B exists, and A and B exist at the same time. In addition, the character " / " in this article generally indicates that the objects before and after are in an "or" logical relationship.

[0065] In the present application, terms such as “first” and “second” are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship of quantity, priority or sequence between these entities or operations.

[0066] Without further limitations, in this application, the words "include", "comprises", "has" or other similar expressions used in the sentences are intended to cover non-exclusive inclusion. These expressions do not exclude the presence of additional elements in the process, method or product including the elements, so that the process, method or product including a series of elements may include not only those limited elements, but also other elements not explicitly listed, or also include elements inherent to such process, method or product.

[0067] Similar to the understanding in the Examination Guidelines, in this application, expressions such as "greater than", "less than", "exceed" and the like are understood to exclude the number itself; expressions such as "above", "below", "within" and the like are understood to include the number itself. In addition, in the description of the embodiments of this application, "multiple" means more than two (including two), and similar expressions related to "multiple" are also understood in this way, such as "multiple groups", "multiple times", etc., unless otherwise clearly and specifically limited.

[0068] In the description of the embodiments of the present application, space-related expressions used, such as "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "vertical", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise", "axial", "radial", "circumferential", etc., indicate the orientation or position relationship based on the orientation or position relationship shown in the specific embodiments or drawings, and are only for the convenience of describing the specific embodiments of the present application or facilitating the reader's understanding, and do not indicate or imply that the referred device or component must have a specific position, a specific orientation, or be constructed or operated in a specific orientation, and therefore cannot be understood as a limitation on the embodiments of the present application.

[0069] Unless otherwise expressly specified or limited, in the description of the embodiments of the present application, the terms such as "install", "connect", "connect", "fix", "set", etc. used should be understood in a broad sense. For example, the "connection" can be a fixed connection, a detachable connection, or an integrated setting; it can be a mechanical connection, an electrical connection, or a communication connection; it can be a direct connection, or it can be indirectly connected through an intermediate medium; it can be the internal connection of two elements or the interaction relationship between two elements. For technicians in the technical field to which the present application belongs, the specific meanings of the above terms in the embodiments of the present application can be understood according to the specific circumstances.

[0070] See also Figure 1 This embodiment provides a robot video processing method, which can satisfy the robot to use one camera to realize multiple video scenes at the same time. The method includes the following steps:

[0071] Step S110: Start the video main thread to collect the original video stream in real time through the camera;

[0072] Step S120: creating a preset number of data channels according to the number of data sub-threads;

[0073] Step S130: when the data sub-thread function is started, the corresponding data channel is opened to copy the original video stream;

[0074] Step S140: the data sub-thread converts the received copied original video stream into required video data and then performs video processing.

[0075] When the robot is using multiple video scenes, the video main thread is started to collect the original video stream in real time through the camera; a preset number of data channels are created according to the number of data sub-threads. When one or several video scenes need to be used, the data sub-threads corresponding to the required video scenes are started. The data sub-threads copy the original video stream collected by the camera through the corresponding data channels, and convert it into the required video data for video processing. Only one camera is needed to meet the application of multiple video scenes at the same time, without adding additional cameras and without increasing additional costs.

[0076] In some embodiments, the data channel includes a reserved data channel. By setting the reserved data channel, when the robot needs to add a new video scene application, after adding a new data sub-thread, the reserved data channel can be used to obtain the original video stream and perform the new video scene application.

[0077] See also Figure 2 In some embodiments, a robot video processing method specifically includes the following steps:

[0078] Step S210: during initialization, a scheduling container is created to store video data required by different data sub-threads;

[0079] Step S210: when receiving a video data request from a user, determining whether the scheduling container is in an idle state;

[0080] If the scheduling container is in an idle state, step S230 is executed: starting the video main thread to collect the original video stream in real time;

[0081] Step S240: creating a preset number of data channels according to the number of data sub-threads;

[0082] Step S250: Start the data sub-thread corresponding to the video data requested by the user, copy the original video stream to the scheduling container through the corresponding data channel and convert it into the corresponding type of video data, and set the video data to the working mode;

[0083] If the scheduling container is not in an idle state, step S260 is executed: determining whether the video data requested by the user is in a working state;

[0084] If the video data requested by the user is not in the working state, step S270 is executed: the data sub-thread corresponding to the video data requested by the user is started, the original video stream is copied to the scheduling container through the corresponding data channel and converted into the corresponding type of video data, and the video data is set to the working mode;

[0085] If the video data requested by the user is in a working state, step S280 is executed: the running state of the video data is fed back to the user.

[0086] When the system is initialized, a container is created first. This container is used to store four types of video data: face recognition, remote video, driving record, and photo capture, and whether they are running; the initial default is idle mode;

[0087] When the system receives a user request, it first determines the state of the scheduling container. If all are in idle mode, it means that the entire video system is in idle state. The first step is to start the video processing main thread, and then start the data sub-thread corresponding to the user's requested video data. After successful startup, the corresponding video data in the scheduling container is set to working mode at the same time, and the success status is fed back to the user. The user can wait for the video data output in the specified data channel.

[0088] If it is determined that at least one type of video data in the scheduling container is in working mode, then it is determined whether the video data corresponding to the data type requested by the user in the container is in working mode; if the video data of this data type in the container is in idle mode, then the data sub-thread corresponding to the video data requested by the user is started, and after successful startup, the video data of the corresponding data type in the container is set to working mode at the same time, and the success status is fed back to the user, and the user can wait for data output in the designated channel;

[0089] If it is determined that at least one type of data in the container is in working mode, and at the same time, the video data of the data type corresponding to the user's request in the container is also in working mode, it indicates that the data requested by the user is being processed, and the running status is directly fed back to the user;

[0090] This video service scheduling subsystem supports users to request the above four types of video data at the same time. These four types of video data can work independently at the same time without interfering with each other. In some embodiments, the data sub-threads include face recognition thread, remote video thread, driving record thread and photo capture thread.

[0091] In some embodiments, the original video stream is in YUV format;

[0092] The “data sub-thread converts the received copied original video stream into the required video data and then performs video processing” specifically includes the following steps:

[0093] After receiving the copied original video stream, the face recognition thread directly uses the original video stream as the required video data and sends it to the face recognition module;

[0094] After receiving the copied original video stream, the remote video thread converts the original video stream in YUV format into video data in H264 format after data format processing, and sends it to the service platform;

[0095] After receiving the copied original video stream, the driving record thread converts the original video stream in YUV format into video data in H264 format after data format processing, and performs MP4 encapsulation processing on the video data in H264 format to obtain video data in MP4 format, and stores the obtained video data in MP4 format;

[0096] After the photo capture thread receives the copied original video stream, it performs JPG encoding processing to obtain a JPG format image.

[0097] Face recognition thread: Face recognition uses YUV data by default. After obtaining the video data of the original video stream, this thread calls back to the face recognition module in real time for use by the face algorithm.

[0098] Remote video thread: The data used by remote video is H264 data. After the remote video thread obtains the original data in YUV format, it undergoes data format processing (the video format is converted to the encoding requirement format, such as NV21 format to I420 format), data scaling processing (to meet users' different resolution requirements), encoding processing (compressing the original data, that is, encoding the I420 format to H264 format), and data sending processing (sending to the platform via RTP);

[0099] Driving record thread: The default data format of driving record is MP4. After the driving record thread obtains the original video stream in YUV format, it undergoes data format processing (the video format is converted to the encoding requirement format, such as NV21 format to I420 format), data scaling processing (to meet the user's different resolution requirements), encoding processing (compressing the original data, that is, encoding the I420 format to H264 format), and MP4 encapsulation processing (encapsulating H264 data into MP4);

[0100] Photo capture thread: The default data format for photo capture is JPG. After the photo capture thread obtains the original video stream in YUV format, it undergoes JPG encoding processing (YUV data is encoded into JPG) to obtain a JPG image.

[0101] In some embodiments, the robot video processing method is implemented by a video data processing subsystem and a video service scheduling subsystem.

[0102] Video data processing subsystem: This subsystem includes two core modules: the video main thread and four data processing sub-threads;

[0103] See also Figure 3 Schematic diagram of the workflow of the video data processing subsystem, video main thread: mainly includes video data acquisition and data distribution;

[0104] Video data acquisition: real-time acquisition of high-resolution (1920x1080 or 1280x720) YUV raw video stream from the camera;

[0105] Data distribution: 5 data channels are created by default, 1 data channel is reserved, and the other 4 data channels are used for face recognition, remote video, driving records, and photo capture;

[0106] When a data sub-thread function is enabled, the data distribution module will start to copy a copy of the original data in real time and distribute it to the corresponding sub-thread for processing;

[0107] When a data sub-thread is closed, the corresponding data distribution channel will also be closed accordingly;

[0108] 4 data sub-threads: face recognition data thread, remote video thread, driving record thread, and photo capture thread.

[0109] Video service dispatch subsystem: This system supports the dispatch of video data such as face recognition, remote video, driving records, and photo capture;

[0110] See also Figure 4 The workflow diagram of the video service scheduling subsystem. When the video service scheduling subsystem is initialized, a container will be created first. This container is used to store four types of video data: face recognition, remote video, driving record, and photo capture. The status of whether the video data is running or not is set to idle mode by default.

[0111] When the system receives a request for class A video data from a user, it first determines the state of the scheduling container. If all are in idle mode, it indicates that the entire video system is in idle mode. The first step is to start the video processing main thread, and then start the data sub-thread corresponding to the user's requested video data. After successful startup, the corresponding class A video data in the scheduling container is set to working mode at the same time, and the success status is fed back to the user. The user can wait for the video data output on the specified data channel.

[0112] If it is determined that at least one type of video data in the scheduling container is in working mode, then it is determined whether the Class A video data corresponding to the user's request in the container is in idle state; if the video data of this data type in the container is in idle mode, then the data sub-thread corresponding to the Class A video data requested by the user is started. After the startup is successful, the video data of the Class A data type in the container is set to working mode at the same time, and the success status is fed back to the user, and the user can wait for data output in the designated channel.

[0113] If it is determined that at least one type of data in the container is in working mode, and at the same time, the class A video data corresponding to the user's request in the container is also in working mode, it indicates that the data requested by the user is being processed, and the running status is directly fed back to the user;

[0114] The video service scheduling subsystem supports users to request the above four types of video data at the same time. The four types of video data can work independently at the same time without interfering with each other.

[0115] Robot video business scenarios:

[0116] The main robot-based video service scenarios include: face recognition, remote video, driving records, photo capture, etc.

[0117] Application scenario introduction:

[0118] Face recognition application: When the welcoming service function is enabled, the robot will automatically walk to the welcoming point at the set time to greet guests. The welcoming service mainly includes welcoming greetings, left and right face tracking, up and down face tracking, and active temperature measurement when a person is recognized. When a user approaches during the robot's welcoming process, the robot can analyze the user's gender, age and other characteristics to proactively make an accurate greeting; the robot can also adjust the head height and posture angle in real time according to the user's face to ensure that the machine screen is always facing the user, improving the user experience; when the robot measures the temperature, it will first perform a liveness verification through face recognition to ensure the validity of the temperature measurement object;

[0119] Face recognition scheduling: After the welcome begins, the application will automatically request face recognition data from the video service scheduling system. After receiving the request, the video service scheduling system will start a subthread of the face recognition data type. After a series of processing, this subthread will eventually send the data to the face recognition algorithm module. After receiving the data, the face recognition module will first analyze and filter the video data to remove video data without faces. Then, the algorithm logic processing of face detection or face recognition will be performed according to the business type.

[0120] Remote video application: There are two main application modes of remote video: active sending and passive request;

[0121] Active sending: When users need assistance from back-end staff, they can make a one-click video call on the robot screen. After the back-end receives the video request, it can conduct a one-to-one video communication with the user, thereby improving service efficiency and quality;

[0122] Passive request: If the robot encounters a special abnormality and cannot drive normally, the customer can choose one-click remote assistance. When the back-end staff receives the request, the cloud server will initiate a video request to the front-end robot. After the front-end robot receives the video request, it will send the video to the cloud server. The back-end staff can remotely control the robot to a designated safe area through the real-time video transmitted back by the robot.

[0123] Remote video scheduling: Whether it is active or passive, after the robot receives a remote video request, the application will automatically request video data from the video service scheduling system. After receiving the request, the video service scheduling system will start a sub-thread of the remote video data type. After a series of processing, this sub-thread will eventually send the data to the cloud server. After receiving the video data, the cloud server will decode and display it;

[0124] Driving record application: When the robot is deployed in a complex scene with a large flow of people, customers can enable the driving record function; if an emergency occurs, the robot video record can be retrieved at any time, which helps to restore the scene when the abnormality occurred;

[0125] Driving record scheduling: After driving record is enabled, the application will automatically request the video data of driving record from the video service scheduling system. After receiving the request, the video service scheduling system will start the sub-thread of remote driving record type. After a series of processing, the real-time data of this sub-thread will be written into an MP4 file every 10 minutes. To store more driving records, a TF memory card can be inserted into the robot screen to increase the storage capacity. Customers can filter the video records by day or time period on the robot driving record function module, and they can play them at any time after clicking.

[0126] Photo capture application: After the customer authorizes the permission to take photos of abnormal events, if the robot detects abnormal conditions such as abnormal body temperature of the user or collision of the autonomous driving, it can automatically capture real-time photos and upload them to the cloud platform to facilitate the investigation and tracing of subsequent problems;

[0127] Photo capture scheduling: When an abnormal event is triggered, the application will automatically request photo data from the video service scheduling system; after receiving the request, the video service scheduling system will start a sub-thread of the photo capture type. After a series of processing, this sub-thread will directly save it as a jpg image and finally send the image to the cloud server.

[0128] See also Figure 5 In another embodiment, a storage medium 510 stores a computer program, and the computer program is executed by a processor 520 to perform the following steps:

[0129] Start the video main thread to collect the original video stream in real time through the camera;

[0130] Create a preset number of data channels according to the number of data sub-threads;

[0131] When the data sub-thread function is started, the corresponding data channel is opened, the original video stream is copied, and sent to the data sub-thread through the corresponding data channel;

[0132] The data sub-thread converts the received copied original video stream into the required video data and then performs video processing.

[0133] When the robot is using multiple video scenes, the video main thread is started to collect the original video stream in real time through the camera; a preset number of data channels are created according to the number of data sub-threads. When one or several video scenes need to be used, the data sub-threads corresponding to the required video scenes are started. The data sub-threads copy the original video stream collected by the camera through the corresponding data channels, and convert it into the required video data for video processing. Only one camera is needed to meet the application of multiple video scenes at the same time, without adding additional cameras and without increasing additional costs.

[0134] In some embodiments, the data channel comprises a reserved data channel.

[0135] In some embodiments, when the computer program is executed by a processor, the following steps are specifically performed:

[0136] During initialization, a scheduling container is created to store the video data required by different data sub-threads;

[0137] When receiving a video data request from a user, determine whether the scheduling container is in an idle state;

[0138] If the scheduling container is in idle state, the video main thread is started to collect the original video stream in real time;

[0139] Create a preset number of data channels according to the number of data sub-threads;

[0140] Start the data sub-thread corresponding to the video data requested by the user, copy the original video stream to the scheduling container through the corresponding data channel and convert it into the corresponding type of video data, and set the video data to the working mode;

[0141] If the scheduling container is not in an idle state, determining whether the video data requested by the user is in a working state;

[0142] If the video data requested by the user is not in working state, the data sub-thread corresponding to the video data requested by the user will be started, the original video stream will be copied to the scheduling container through the corresponding data channel and converted into the corresponding type of video data, and the video data will be set to working mode;

[0143] If the video data requested by the user is in a working state, the running state of the video data is fed back to the user.

[0144] In some embodiments, the data sub-threads include a face recognition thread, a remote video thread, a driving record thread, and a photo capture thread.

[0145] In some embodiments, the original video stream is in YUV format;

[0146] The “data sub-thread converts the received copied original video stream into the required video data and then performs video processing” specifically includes the following steps:

[0147] After receiving the copied original video stream, the face recognition thread directly uses the original video stream as the required video data and sends it to the face recognition module;

[0148] After receiving the copied original video stream, the remote video thread converts the original video stream in YUV format into video data in H264 format after data format processing, and sends it to the service platform;

[0149] After receiving the copied original video stream, the driving record thread converts the original video stream in YUV format into video data in H264 format after data format processing, and performs MP4 encapsulation processing on the video data in H264 format to obtain video data in MP4 format, and stores the obtained video data in MP4 format;

[0150] After the photo capture thread receives the copied original video stream, it performs JPG encoding processing to obtain a JPG format image.

[0151] Finally, it should be noted that although the above embodiments have been described in the specification and drawings of this application, this does not limit the scope of patent protection of this application. All technical solutions generated by replacing or modifying equivalent structures or equivalent processes based on the essential concept of this application using the contents recorded in the specification and drawings of this application, as well as directly or indirectly implementing the technical solutions of the above embodiments in other related technical fields, are included in the scope of patent protection of this application.

Claims

1. A robot video processing method, characterized in that: The following steps are involved: During initialization, a scheduling container is created to store the video data required by different data sub-threads; When receiving a video data request from a user, it determines whether the scheduling container is idle, supports users to request four types of video data at the same time, and the four types of video data can work independently at the same time; If the scheduling container is in idle state, the main video thread is started to collect the original video stream in real time through the camera; Creating a preset number of data channels according to the number of data sub-threads, wherein the data sub-threads include a face recognition thread, a remote video thread, a driving record thread, and a photo capture thread; Start the data sub-thread corresponding to the video data requested by the user, copy the original video stream to the scheduling container through the corresponding data channel and convert it into the corresponding type of video data, and set the video data to the working mode; If the scheduling container is not in an idle state, determining whether the video data requested by the user is in a working state; If the video data requested by the user is not in working state, the data sub-thread corresponding to the video data requested by the user is started, the original video stream is copied to the scheduling container through the corresponding data channel and converted into the corresponding type of video data, and the video data is set to working mode; If the video data requested by the user is in a working state, the running state of the video data is fed back to the user.

2. The robot video processing method according to claim 1, characterized in that: The data channel includes a reserved data channel.

3. The robot video processing method according to claim 1, characterized in that: The original video stream is in YUV format; The data sub-thread converts the received copied original video stream into the required video data and then performs video processing, which specifically includes the following steps: After receiving the copied original video stream, the face recognition thread directly uses the original video stream as the required video data and sends it to the face recognition module; After receiving the copied original video stream, the remote video thread converts the original video stream in YUV format into video data in H264 format after data format processing, and sends it to the service platform; After receiving the copied original video stream, the driving record thread converts the original video stream in YUV format into video data in H264 format after data format processing, and performs MP4 encapsulation processing on the video data in H264 format to obtain video data in MP4 format, and stores the obtained video data in MP4 format; After the photo capture thread receives the copied original video stream, it performs JPG encoding processing to obtain a JPG format image.

4. A storage medium storing a computer program, characterized in that: The computer program is executed by a processor to perform the following steps: During initialization, a scheduling container is created to store the video data required by different data sub-threads; When receiving a video data request from a user, it determines whether the scheduling container is idle, supports users to request four types of video data at the same time, and the four types of video data can work independently at the same time; If the scheduling container is in idle state, the main video thread is started to collect the original video stream in real time through the camera; Creating a preset number of data channels according to the number of data sub-threads, wherein the data sub-threads include a face recognition thread, a remote video thread, a driving record thread, and a photo capture thread; Start the data sub-thread corresponding to the video data requested by the user, copy the original video stream to the scheduling container through the corresponding data channel and convert it into the corresponding type of video data, and set the video data to the working mode; If the scheduling container is not in an idle state, determining whether the video data requested by the user is in a working state; If the video data requested by the user is not in working state, the data sub-thread corresponding to the video data requested by the user is started, the original video stream is copied to the scheduling container through the corresponding data channel and converted into the corresponding type of video data, and the video data is set to working mode; If the video data requested by the user is in a working state, the running state of the video data is fed back to the user.

5. The storage medium according to claim 4, characterized in that: The data channel includes a reserved data channel.

6. The storage medium according to claim 4, characterized in that: The original video stream is in YUV format; The data sub-thread converts the received copied original video stream into the required video data and then performs video processing, which specifically includes the following steps: After receiving the copied original video stream, the face recognition thread directly uses the original video stream as the required video data and sends it to the face recognition module; After receiving the copied original video stream, the remote video thread converts the original video stream in YUV format into video data in H264 format after data format processing, and sends it to the service platform; After receiving the copied original video stream, the driving record thread converts the original video stream in YUV format into video data in H264 format after data format processing, and performs MP4 encapsulation processing on the video data in H264 format to obtain video data in MP4 format, and stores the obtained video data in MP4 format; After the photo capture thread receives the copied original video stream, it performs JPG encoding processing to obtain a JPG format image.

Images