Data processing method and device, electronic equipment and vehicle
By matching and labeling the human key point external frame data and human body external frame data, a second sample image collection for training was established, which solved the problem of low accuracy in inference in the prior art human key point detection model, and achieved more High accuracy of detection results.
Patent Information
- Application Number
- CN202310511630.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-08
- Publication Date
- 2024-11-08
AI Technical Summary
In the prior art, when using human body external frame data as model input data for detection of key points in human body, the accuracy of the detection results is low.
By obtaining the sample image collection, extracting and matching the data of the human key points and the data of the human body's external frame, annotating the matching data, and establishing a second sample image collection for training, so that the data input during training and the input during inference during inference The data are consistent, thereby improving the accuracy of the detection results.
The accuracy of the human key point detection model in the inference stage is improved, ensuring that the model can better learn and recognize features during training and reasoning.
Smart Images

Figure CN118918406A_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of data processing technology, and in particular to a data processing method, device, electronic equipment and vehicle. Background Art
[0002] With the continuous development of computer vision technology, human key point detection, as one of the basic algorithms of computer vision, has also been widely used. The importance of human key point detection in intelligent video surveillance, patient monitoring systems, human-computer interaction, virtual reality, human animation, smart home, smart security, and athlete auxiliary training is becoming more and more obvious.
[0003] At present, the training method of the human key point detection model is mainly based on the human key point external frame data for model training. Specifically, the human key point external frame data is sent into the network, and the human key point detection network is trained.
[0004] However, the human body key point detection model trained in this way has poor performance on the input of human body bounding box data. During the reasoning process, when the human body bounding box data is used as the model input data for human body key point detection, the accuracy of the human body key point detection results will be low. Summary of the invention
[0005] In view of this, the present application provides a data processing method, device, system and electronic device, the main purpose of which is to improve the technical problem that the accuracy of human key point detection results is low when the current existing technology uses human body external frame data as model input data for human body key point detection.
[0006] In a first aspect, the present application provides a data processing method, comprising:
[0007] Acquire a first sample image set;
[0008] Processing the sample images in the first sample image set to obtain human body key point circumscribed frame data and human body circumscribed frame data in the sample images;
[0009] Matching the human body key point external frame data with the human body external frame data;
[0010] In the sample image, the human body bounding box data matching the human body key point bounding box data is annotated;
[0011] A second sample image set is established by using the labeled sample images, and the second sample image set is used for training a human key point detection model.
[0012] In a second aspect, the present application provides a data processing device, comprising:
[0013] An acquisition module, configured to acquire a first sample image set;
[0014] A processing module is configured to process the sample images in the first sample image set to obtain the human body key point circumscribed frame data and the human body circumscribed frame data in the sample images;
[0015] A matching module is configured to match the human body key point external frame data with the human body external frame data;
[0016] The annotation module is configured to annotate the human body external bounding box data matched by the human body key point external bounding box data in the sample image;
[0017] The establishment module is configured to establish a second sample image set using the labeled sample images, and the second sample image set is used for training a human key point detection model.
[0018] In a third aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method described in the first aspect is implemented.
[0019] In a fourth aspect, the present application provides an electronic device, comprising a storage medium, a processor, and a computer program stored on the storage medium and executable on the processor, wherein the processor implements the method described in the first aspect when executing the computer program.
[0020] In a fifth aspect, the present application provides a vehicle, comprising: the device as described in the second aspect, or the electronic device as described in the fourth aspect.
[0021] By means of the above technical scheme, the present application provides a data processing method, device, system and electronic device, firstly obtaining a first sample image set; then processing the sample images in the first sample image set to obtain the human body key point external frame data and the human body external frame data in the sample image; then matching the human body key point external frame data and the human body external frame data; then marking the human body external frame data matched by each human body key point external frame data in the sample image; and then using the marked sample images to establish a second sample image set, the second sample image set is used for training the human body key point detection model. Compared with the current prior art, the present application aligns the human body key point external frame data and the human body external frame data in the data processing stage, and changes the original input data (human body key point external frame data) of the human body key point detection model during training to data consistent with the human body external frame data type input by the human body key point detection model during reasoning, so that the network model can better learn the features required in the reasoning stage during the training process. During model reasoning (using the human body external frame data as the model input data for human body key point detection), the accuracy of the human body key point detection results can be improved.
[0022] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.
[0024] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0025] Figure 1 A schematic diagram of a data processing method provided in an embodiment of the present application is shown;
[0026] Figure 2 A schematic diagram of a process flow of a human body detection network training method provided in an embodiment of the present application is shown;
[0027] Figure 3 A schematic diagram of a flow chart of a human key point detection network training method provided in an embodiment of the present application is shown;
[0028] Figure 4A schematic diagram showing a flow chart of another data processing method provided in an embodiment of the present application is shown;
[0029] Figure 5 An exemplary flow chart of a data processing method provided by an embodiment of the present application is shown;
[0030] Figure 6 A structural schematic diagram of a data processing device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION
[0031] In order to more clearly understand the above-mentioned purposes, features and advantages of the present application, the scheme of the present application will be further described below. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.
[0032] In order to improve the technical problem that the accuracy of the key point detection result of the human body is low when the human body external frame data is used as the model input data for human body key point detection in the existing technology, this embodiment provides a data processing method, such as Figure 1 As shown, the method includes:
[0033] Step 101: Obtain a first sample image set.
[0034] The first sample image set may be images of the interior of the cabin captured by an in-vehicle camera. The camera captures a large number of images of the interior of the cabin, and the captured images are used to form a set, which is the first sample image set.
[0035] The cockpit refers to the interior space of a car, which is the compartment for the driver and passengers.
[0036] In an embodiment of the present application, obtaining the first sample image set can specifically be that a camera in the vehicle cabin collects a large number of images inside the cabin. The images in the image set can be images of the driver, images of passengers, images of the cabin environment, images of the vehicle's current external environment, etc. The image set may include one or more of the above-mentioned images, and the types of images in the image set are not specifically limited herein.
[0037] Step 102: Process the sample images in the first sample image set to obtain the human body key point circumscribed frame data and the human body circumscribed frame data in the sample images.
[0038] The key points of the human body refer to the key position points of various parts of the human body. The key points of the human body are crucial for describing human posture and predicting human behavior. The key points of the human body mainly include three points of the left and right arms (wrist, elbow, shoulder), three points of the left and right legs (ankle, knee, hip), hip and buttocks points, and head points (eyes, chin, top of the head).
[0039] The human key point data may include the position information (such as coordinate information, etc.), attribute information, etc. of the above-mentioned human key points in the sample image, such as recording the coordinate information and attribute information (visible or invisible) of the key points to obtain the human key point data. The human key point bounding box refers to finding a minimum rectangle that can contain all human key points with visible attributes. The rectangle is the key point bounding box of this batch of human key points. Human data refers to the annotation of the human body bounding box of the image containing the human body, and recording the coordinate information of the bounding box.
[0040] Image processing refers to the technology of using computers to analyze images to achieve the desired results, also known as image processing. Image processing generally refers to digital image processing. A digital image refers to a large two-dimensional array obtained by shooting with industrial cameras, video cameras, scanners and other equipment. The elements of the array are called pixels, and their values are called grayscale values. Image processing technology generally includes image compression, enhancement, restoration, matching, description and recognition.
[0041] In an embodiment of the present application, the sample images in the first sample image set are processed, and specifically, data can be extracted from each sample image in the acquired first sample image set to obtain human body data and human body key point data, and then an external frame extraction is performed on the acquired human body data and human body key point data to obtain the above-mentioned human body key point external frame data and human body external frame data.
[0042] In an embodiment of the present application, the acquired human body data may specifically be human body data obtained by annotating an image containing a human body with a human body external frame; the human body key point data may specifically be human body key point data obtained by annotating an image containing a human body with a human body key point external frame; accordingly, the human body external frame data may specifically be human body external frame data obtained by extracting the human body external frame from the above-mentioned annotated human body data; the human body key point external frame data may specifically be human body key point external frame data obtained by extracting the key point external frame from the above-mentioned annotated human body key point data.
[0043] Optionally, the human body data specifically refers to the coordinate information of the human body external frame obtained based on the human body data; the human body key point data specifically refers to the coordinate information of the human body key point external frame obtained based on the human body key point data; accordingly, the human body external frame data is the minimum rectangle covering all the above external frame coordinate information; the human body key point external frame data is the minimum rectangle covering all the above human body key point external frame coordinate information.
[0044] Step 103: Match the human body key point circumscribed frame data with the human body circumscribed frame data.
[0045] In an embodiment of the present application, the acquired human body key point circumscribed frame data and human body circumscribed frame data are matched. For example, the matching method can be to calculate the intersection-and-union ratio of the above-mentioned human body key point circumscribed frame data and human body circumscribed frame data, and then match them according to the intersection-and-union ratio.
[0046] The intersection over union (IoU) is an algorithm that calculates the overlap ratio of different images and is often used in target detection or semantic segmentation tasks in the field of deep learning.
[0047] Step 104: in the sample image, mark the human body bounding box data that matches the human body key point bounding box data.
[0048] In the embodiment of the present application, if the above-mentioned human body key point circumscribed frame data and human body circumscribed frame data are matched successfully, the human body circumscribed frame data and the pairing information are stored in the human body key point circumscribed frame data of the sample image.
[0049] Step 105: Use the labeled sample images to establish a second sample image set.
[0050] The second sample image set is used for training a human key point detection model.
[0051] The human key point detection network corresponding to the human key point detection model refers to a deep learning network built for the detection scene. It is usually composed of a backbone network (backbone), a shoulder network (neck), and a head network (head), which is used to detect whether there are human key points and return the coordinates of the human key points. Correspondingly, the human body detection network (corresponding to the human body detection model) refers to a deep learning network built for the detection scene. It is usually composed of a backbone network, a shoulder network, and a head network, which is used to detect whether there is a human body and return the coordinates of the human body external frame.
[0052] In order to realize specific functions inside the cockpit, it is usually necessary to train a human body detection network and a human body key point detection network and apply them, including training a human body detection network and a human body key point detection network respectively, wherein the human body detection network is trained based on human body data, such as Figure 2 As shown in Figure 1, based on the human key point data, the human key point detection network is trained. Before training the human key point detection network, it is necessary to first obtain the key point external frame based on the human key points with visible attributes, and then cut out part of the image of the external frame and send it to the human key point detection network for training, as shown in Figure 1. Figure 3As shown. Specifically, when running a specific function inside the cockpit, it is necessary to first send the original image to the human body detection network to obtain the human body external frame, and then cut out part of the image of the human body external frame and send it to the human body key point detection network to obtain the coordinate position of the human body key points. In this process, the problem of model training and reasoning are not aligned. The main reason is: when training the human body key point detection network, the key point external frame is sent to the network, and when reasoning the human body key point detection model, the human body external frame obtained by the human body detection network is sent to the network. There are certain differences between these two external frames. The human body key point detection model obtained through training is aimed at the input based on the key point external frame, and the performance of the input such as the human body external frame is poor.
[0053] In an embodiment of the present application, a second sample image set is established using the labeled sample images, and the second sample image set may include the data that has been successfully matched and labeled in the above steps. For an embodiment of the present application, the second sample image set is used for training the human key point detection model. Compared with the current prior art, this embodiment aligns the human key point external frame data and the human body external frame data in the data processing stage, and changes the original input data of the human body key point detection model during training (human body key point external frame data) to data consistent with the human body external frame data type input by the human body key point detection model during reasoning, so that the network model can better learn the features required in the reasoning stage during the training process. During model reasoning (using the human body external frame data as model input data for human body key point detection), the accuracy of the human body key point detection results can be improved.
[0054] Further, as a refinement and extension of the above embodiment, in order to fully illustrate the specific implementation process of the method of the embodiment of the present disclosure, the embodiment of the present application provides the following Figure 4 The specific method shown includes:
[0055] Step 201: Obtain a first sample image set.
[0056] Optionally, the first sample image set may be an image set captured by an in-vehicle camera, including but not limited to driver images, passenger images, vehicle cabin environment images, and the like.
[0057] For example, taking vehicle A as an example, vehicle A is equipped with an in-vehicle camera, which captures images of people and the environment in the vehicle in real time while vehicle A is driving. The collection of a large amount of image data collected by vehicle A is the collection of the first sample images obtained.
[0058] Step 202: Process the sample images in the first sample image set to obtain the human body key point circumscribed frame data and the human body circumscribed frame data in the sample images.
[0059] In some embodiments, step 202 may specifically include: filtering sample images that do not contain human features in the first sample image set; processing the sample images in the filtered first sample image set to obtain human key point external frame data and human external frame data in the sample images. In this way, the data quality of the data set required for training the body key point detection model can be improved, and the interference of invalid data on model training can be reduced.
[0060] Exemplarily, based on step 201, the first sample image set includes a driver image in vehicle A, a passenger image in vehicle A, an environment image in vehicle A, an external environment image of vehicle A, etc. After obtaining the first sample image set of vehicle A, the first sample image of vehicle A is filtered to obtain an image that can be further processed, which may be a set of images containing human body information, such as the driver image in vehicle A and the passenger image in vehicle A. Taking picture a in the set as an example, for the original picture a, based on the key points with visible attributes in the human body key point data, the key point external frames of all the human bodies therein are obtained (one human body corresponds to one external frame). Specifically, the original picture a is cropped to obtain the coordinate information of the external frames of the human body key points in the original picture a, and the coordinate information of the external frames of all the human body key points cropped in the original picture a is collected to obtain the minimum rectangle of the human body key points, which is the human body key point external frame data in picture a.
[0061] Correspondingly, for the original picture a, based on the human body data, the human body bounding box of all human bodies is extracted, specifically, the original picture a is cropped to obtain the coordinate information of the human body bounding box in the original picture a, and then extracted to obtain the coordinate information of the human body bounding box based on the original picture a.
[0062] Step 203: Obtain the intersection-over-union ratio of the human body key point circumscribed frame data and the human body circumscribed frame data.
[0063] In some embodiments, step 203 may specifically include: respectively obtaining the intersection information and union information of the human body key point external frame data and the human body external frame data; performing ratio processing on the intersection information and the union information to obtain the intersection-union ratio of the human body key point external frame data and the human body external frame data.
[0064] Exemplarily, based on step 202, the intersection information and union information of the human body key point external frame data and the human body external frame data based on picture a are obtained, and the above intersection information and union information are ratio processed to obtain the intersection-union ratio of the human body key point external frame data and the human body external frame data based on picture a.
[0065] Step 204 , determine whether the intersection-over-union ratio of the human body key point circumscribed frame data and the human body circumscribed frame data is greater than or equal to a preset threshold, and if so, determine that the match is successful.
[0066] Exemplarily, based on step 203, the ratio of the intersection-and-union ratio of the human body key point circumscribed frame data and the human body circumscribed frame data based on image a is obtained and compared with a preset intersection-and-union ratio threshold to determine whether the intersection-and-union ratio of the human body key point circumscribed frame data and the human body circumscribed frame data of image a is greater than or equal to the preset threshold. If so, the match is determined to be successful; if not, the match is determined to have failed, and then the next one is switched for matching.
[0067] In some examples, the method of this embodiment also includes: if there are multiple human body external frame data and their intersection-and-union ratios with the human body key point external frame data are greater than or equal to a preset threshold, then from the multiple human body external frame data, a human body external frame data with the largest intersection-and-union ratio value is selected as the human body external frame data that matches the human body key point external frame data.
[0068] Exemplarily, based on picture a, if there are multiple human body external frame data and the intersection and union ratios of the human body key point external frame data are greater than or equal to a preset threshold, the ratios of the multiple intersection and union ratios are compared, and from the multiple human body external frame data, a human body external frame data with the largest intersection and union ratio value is selected as the human body external frame data that matches the human body key point external frame data.
[0069] Step 205: In the sample image, mark the human body bounding box data that matches the human body key point bounding box data.
[0070] Exemplarily, based on step 204, if the human body key point circumscribed frame data and the human body circumscribed frame data based on image a are matched successfully and there is only one data greater than a preset intersection-and-union ratio threshold, then the human body circumscribed frame coordinate information of the successful match is directly marked in the human body key point circumscribed frame coordinate information; if the human body key point circumscribed frame data and the human body circumscribed frame data based on image a are matched successfully, but there are multiple human body circumscribed frame data and their intersection-and-union ratios with the human body key point circumscribed frame data are greater than or equal to the preset threshold, then the largest group of data in the data is obtained, and its human body circumscribed frame coordinate information is marked in the human body key point circumscribed frame coordinate information.
[0071] Optionally, if the human body key point circumscribed frame data based on picture a and the human body circumscribed frame data fail to match, the coordinate information of the human body key point circumscribed frame based on picture a is directly obtained.
[0072] Step 206: Use the labeled sample images to establish a second sample image set.
[0073] The second sample image set is used for training a human key point detection model.
[0074] In some embodiments, in order to enrich the number of sample images, step 206 may specifically include: using an image enhancement algorithm to transform the labeled sample images; and then using the labeled sample images and the expanded images obtained after the transformation to establish a second sample image set. In this way, the second sample image set can be effectively expanded, and the model obtained by training the expanded second sample image set can improve the accuracy of model recognition.
[0075] The image enhancement algorithm includes at least one or more combinations of scaling, shearing, rotation, occlusion, beveling, affine transformation, distortion transformation, channel swapping, and black-and-white conversion.
[0076] Exemplarily, based on step 205, if the match in step 205 is successful and data annotation is performed, the annotated sample image is transformed using an image enhancement algorithm, wherein the image enhancement algorithm includes at least one or more combinations of scaling, shearing, rotation, occlusion, beveling, affine transformation, distortion transformation, channel swapping, and black and white conversion, and a second sample image set is established using the annotated sample image and the expanded image obtained after the transformation, and the data in the second sample image set is used to train a human key point detection model.
[0077] Further, in order to fully illustrate the specific implementation process of the method of this embodiment, this embodiment provides the following examples, but is not limited thereto: Figure 5 The flowchart of the example is shown.
[0078] Taking vehicle B as an example, when vehicle B is in normal operation, the camera in vehicle B will collect images of the human body and the environment in vehicle B, filter the image set collected by vehicle B, obtain an image set containing human images, crop the image set, take cropped image b as an example, obtain the coordinate information of the key points of the human body and the human body coordinate information in the cropped image b, collect the coordinate information of the key points of the human body, obtain the external frame data of the key points of the human body containing all the coordinate information of the key points of the human body, extract the human body coordinate information, and obtain the external frame data of the human body. Obtain the intersection information and union information of the external frame data of the key points of the human body and the external frame data of the human body respectively; perform ratio processing on the intersection information and the union information to obtain the intersection and union ratio of the key points of the human body and the external frame data of the human body, and judge whether the intersection and union ratio of the key points of the human body and the external frame data of the human body is greater than or equal to the preset threshold. If so, it is determined that the match is successful, and if not, it is determined that the match fails.
[0079] If the match is successful, and there is only one data greater than the preset intersection-and-union ratio threshold, the coordinate information of the human body external frame that is successfully matched is directly annotated in the coordinate information of the human body key point external frame; if the human body key point external frame data based on image b and the human body external frame data are successfully matched, but there are multiple human body external frame data and the intersection-and-union ratio of the human body key point external frame data is greater than or equal to the preset threshold, then the largest group of data is obtained, and its human body external frame coordinate information is annotated in the human body key point external frame coordinate information. Then, the image enhancement algorithm is used to transform the annotated sample image, wherein the image enhancement algorithm at least includes one or more combinations of scaling, shearing, rotation, occlusion, beveling, affine transformation, distortion transformation, channel interchange, and black and white, and the annotated sample image and the expanded image obtained after the transformation are used to establish a second sample image set, and the data in the second sample image set are used to train the human key point detection model.
[0080] Compared with the current existing technology, this embodiment aligns the human body key point external frame data and the human body external frame data in the data processing stage, and changes the original input data of the human body key point detection model during training (human body key point external frame data) to data consistent with the human body external frame data type input by the human body key point detection model during reasoning, so that the network model can better learn the features required in the reasoning stage during the training process. During model reasoning (using the human body external frame data as model input data for human body key point detection), the accuracy of the human body key point detection results can be improved.
[0081] Further, as Figure 1 and Figure 4 The specific implementation of the method shown in this embodiment provides a data processing device, such as Figure 6 As shown, the device includes: an acquisition module 31, a processing module 32, a matching module 33, a labeling module 34, and a building module 35.
[0082] The acquisition module 31 is configured to acquire a first sample image set.
[0083] The processing module 32 is configured to process the sample images in the first sample image set to obtain the human body key point circumscribed frame data and the human body circumscribed frame data in the sample images.
[0084] The matching module 33 is configured to match the human body key point circumscribed frame data with the human body circumscribed frame data.
[0085] The marking module 34 is configured to mark the human body circumference frame data that matches the human body key point circumference frame data in the sample image.
[0086] The establishing module 35 is configured to establish a second sample image set using the labeled sample images, where the second sample image set is used for training a human key point detection model.
[0087] In a specific application scenario, the matching module 33 is specifically configured to obtain the intersection and union ratio of the human body key point circumscribed frame data and the human body circumscribed frame data; determine whether the intersection and union ratio of the human body key point circumscribed frame data and the human body circumscribed frame data is greater than or equal to a preset threshold, and if so, determine that the match is successful.
[0088] In a specific application scenario, the matching module 33 is further configured to respectively obtain the intersection information and union information of the human body key point external frame data and the human body external frame data; perform ratio processing on the intersection information and the union information to obtain the intersection-union ratio of the human body key point external frame data and the human body external frame data.
[0089] In a specific application scenario, the matching module 33 is further configured to select a human body external frame data with the largest intersection-and-union ratio value from the multiple human body external frame data as the human body external frame data that matches the human body key point external frame data if the intersection-and-union ratios of the multiple human body external frame data with the human body key point external frame data are both greater than or equal to a preset threshold.
[0090] In a specific application scenario, the processing module 32 is specifically configured to filter the sample images in the first sample image set that do not have human features; process the sample images in the first sample image set after filtering to obtain the human key point external frame data and human body external frame data in the sample images.
[0091] In a specific application scenario, the labeling module 34 is specifically configured to transform the labeled sample image using an image enhancement algorithm, wherein the image enhancement algorithm includes at least one or more combinations of scaling, shearing, rotation, occlusion, beveling, affine transformation, distortion transformation, channel swapping, and black and white conversion; and establish the second sample image set using the labeled sample image and the expanded image obtained after its transformation.
[0092] It should be noted that for other corresponding descriptions of the functional units involved in the data processing device provided in this embodiment, reference can be made to Figure 1 and Figure 4 The corresponding description in will not be repeated here.
[0093] Based on the above Figure 1 and Figure 4The method shown in the figure, accordingly, the embodiment of the present disclosure also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the above-mentioned Figure 1 and Figure 4 The method shown.
[0094] Based on this understanding, the technical solution of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.), and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of various implementation scenarios of the present disclosure.
[0095] Based on the above Figure 1 and Figure 4 The method shown, and Figure 6 In order to achieve the above-mentioned purpose, the embodiment of the present disclosure also provides an electronic device, which can be configured on a computer terminal, etc. The device includes a storage medium and a processor; the storage medium is used to store a computer program; the processor is used to execute the computer program to achieve the above-mentioned Figure 1 and Figure 4 The method shown.
[0096] In some embodiments, the above-mentioned physical device may also include a user interface, a network interface, a camera, a radio frequency (RF) circuit, a sensor, an audio circuit, a WI-FI module, etc. The user interface may include a display, an input unit such as a keyboard, etc., and the optional user interface may also include a USB interface, a card reader interface, etc. The network interface may include a standard wired interface, a wireless interface (such as a WI-FI interface), etc. in some embodiments.
[0097] Those skilled in the art will appreciate that the above-mentioned physical device structure provided in the embodiments of the present disclosure does not constitute a limitation on the physical device, and may include more or fewer components, or a combination of certain components, or different arrangements of components.
[0098] The storage medium may also include an operating system and a network communication module. The operating system is a program that manages the hardware and software resources of the above-mentioned physical device, and supports the operation of the information processing program and other software and / or programs. The network communication module is used to realize the communication between the components inside the storage medium, and the communication with other hardware and software in the information processing physical device.
[0099] Based on the above electronic device, the embodiment of the present disclosure further provides a vehicle, which may specifically include: Figure 6The device shown in the figure or the electronic device as described above. The vehicle can be a new energy vehicle or a traditional vehicle.
[0100] Through the description of the above disclosed implementation methods, those skilled in the art can clearly understand that the present disclosure can be implemented by means of software plus a necessary general hardware platform, or can be implemented by hardware. By applying the solution of the embodiment of the present disclosure, the human body key point external frame data and the human body external frame data are aligned in the data processing stage, and the original input data of the human body key point detection model during training (human body key point external frame data) is changed to data consistent with the human body external frame data type used by the human body key point detection model during reasoning, so that the network model can better learn the features required in the reasoning stage during the training process. During model reasoning (using the human body external frame data as the model input data for human body key point detection), the accuracy of the human body key point detection results can be improved.
[0101] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the term "comprises" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.
[0102] The above description is only a specific embodiment of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but will conform to the widest scope consistent with the principles and novel features applied herein.
Claims
1. A data processing method, characterized in that: include: Acquire a first sample image set; Processing the sample images in the first sample image set to obtain human body key point circumscribed frame data and human body circumscribed frame data in the sample images; Matching the human body key point circumscribed frame data with the human body circumscribed frame data; In the sample image, annotate the human body key point circumscribed frame data matching the human body circumscribed frame data; A second sample image set is established by using the labeled sample images, and the second sample image set is used for training a human key point detection model.
2. The method according to claim 1, characterized in that: The matching of the human body key point circumscribed frame data with the human body circumscribed frame data includes: Obtaining the intersection-and-union ratio of the human body key point circumscribed frame data and the human body circumscribed frame data; It is determined whether the intersection-over-union ratio of the human body key point circumscribed frame data and the human body circumscribed frame data is greater than or equal to a preset threshold, and if so, it is determined that the match is successful.
3. The method according to claim 2, characterized in that The obtaining of the intersection-over-union ratio of the human body key point circumscribed frame data and the human body circumscribed frame data comprises: Respectively obtain the intersection information and union information of the human body key point external frame data and the human body external frame data; The intersection information and the union information are ratio-processed to obtain the intersection-and-union ratio of the human body key point circumscribed frame data and the human body circumscribed frame data.
4. The method according to claim 2 or 3, characterized in that: The matching of the human body key point circumscribed frame data with the human body circumscribed frame data also includes: If there are multiple human body external frame data and their intersection-and-union ratios with the human body key point external frame data are all greater than or equal to a preset threshold, then a human body external frame data with the largest intersection-and-union ratio value is selected from the multiple human body external frame data as the human body external frame data that matches the human body key point external frame data.
5. The method according to claim 1, characterized in that The processing of the sample images in the first sample image set to obtain the human body key point external frame data and the human body external frame data in the sample images includes: Filtering sample images without human features in the first sample image set; The filtered sample images in the first sample image set are processed to obtain human body key point circumscribed frame data and human body circumscribed frame data in the sample images.
6. The method according to claim 1, characterized in that The step of establishing a second sample image set by using the labeled sample images includes: Transforming the labeled sample image using an image enhancement algorithm, wherein the image enhancement algorithm at least includes a combination of one or more of scaling, shearing, rotation, occlusion, beveling, affine transformation, distortion transformation, channel swapping, and black-and-white conversion; The second sample image set is established by using the labeled sample images and the expanded images obtained after transformation.
7. A data processing device, characterized in that: The device comprises: An acquisition module, configured to acquire a first sample image set; A processing module is configured to process the sample images in the first sample image set to obtain the human body key point circumscribed frame data and the human body circumscribed frame data in the sample images; A matching module, configured to match the human body key point circumscribed frame data with the human body circumscribed frame data; A labeling module is configured to label the human body external frame data matched by the human body key point external frame data in the sample image; The establishing module is configured to establish a second sample image set using the labeled sample images, wherein the second sample image set is used for training a human key point detection model.
8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.
9. An electronic device comprising a storage medium, a processor, and a computer program stored in the storage medium and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 6 is implemented.
10. A vehicle, characterized in that: include: The device as claimed in claim 7 or the electronic device as claimed in claim 9.
